When using NVENC as the encode implementation for the transcoder, you can leverage a few built-in nvidia command line options that will assist in monitoring gpu usage. Please try the following:
(Windows/Linux)
nvidia-smi -q -d UTILIZATION
(Linux Only)
nvidia-smi dmon
You can check here for further documentation & output.