NVIDIA Nsight Python Tutorial: How to Profile GPU Kernels

This is the available feed excerpt, which may be shortened or in the publisher’s original language.

In this tutorial, we implement on Nsight Python, NVIDIA’s open-source Python interface to its kernel profilers. Instead of running a script under the Nsight Compute command line and digging through a report afterward, we mark the code we care about with nsight.annotate, decorate a benchmark function with nsight.analyze.kernel, and get back a pandas DataFrame of hardware metrics per annotated region, per configuration, and per run. Since version 1.0, the library attaches Nsight Compute to the running Python process, making it usable in Jupyter and Colab, and the upcoming 1.1 release adds an experimental CUPTI backend, explicit tool management, and custom information collectors. We run everything on a free Colab T4: we install a current Nsight Compute next to the one Colab ships, profile a first kernel, decide how many runs a measurement needs, measure what cache and clock control do to the same kernel, sweep matrix sizes in fp32 and fp16 with a derived TFLOP/s metric, use hardware counters to tell compute-bound from memory-bound work, measure kernel fusion in bytes moved, record clocks and temperature per run, compare the CUPTI backend with Nsight Compute, and keep the raw report for deeper analysis. Copy CodeCopiedUse a different Browserimport os import sys import glob import json import time import shutil import textwrap import traceback import subprocess RESULTS = {} def banner(title): print("\n" + "=" * 78) print(title) print("=" * 78) def section(name): def wrap(fn): def run(*a, **kw): banner(name) try: out = fn(*a, **kw) RESULTS[name] = out if isinstance(out, str) else "ok" return out except Exception as e: RESULTS[name] = f"SKIPPED / FAILED -> {type(e).__name__}: {str(e).splitlines()[0][:160]}" print(f"\n[!] {name} did not complete: {type(e).__name__}: {e}") traceback.print_exc(limit=3)