Basketball Player Tracking
A project to track players in broadcast basketball footage and map them onto a top-down court. This is the second round: what I changed, how I checked it, and where I'm stuck.
YOLO · ByteTrack · Homography · OpenCV · Evaluation tooling
Same footage and tracking, before and after. Dot color shows the team.
Iteration 2
What I changed
Team colors
The first version decided each player's team once for the whole clip, so a player could get stuck on the wrong team. I switched to deciding it from their recent jersey colors, frame by frame, comparing the mix of colors in each jersey rather than a single average color. On the clips I tested, far fewer players ended up on the wrong team.
Minimap dots
Dots used to linger where a player was last seen. Now they disappear when the tracker loses the player.
Method
How I'm checking it
I didn't want to judge this by eye, so I built a small labeling tool to hand-mark players and court points on video frames, then score the pipeline against them. So far I've only labeled a handful of clips, so these are early sanity checks, not results. On those, the court mapping looked close and team colors clearly improved. Building a bigger labeled set is part of the next round.
Open problems
Where I'm stuck
- On harder footage, players drop in and out of detection.
- When two players overlap closely, especially on screens at a shallow camera angle, the tracker sometimes swaps their IDs. A pretrained re-identification model didn't help in my tests.
- A jersey half-hidden behind an opponent can still read as the wrong team.
Harder footage: players drop in and out of detection and get confused when they overlap.
Next
What I want to try
Next I want to let the user click one player from each team to set the jersey colors, rather than having the system guess them, and then work on keeping IDs steady through overlaps.
If you've worked on this, I'd love to hear how you've handled ID swaps during occlusions in broadcast sports footage.
Iteration 1
V1 — the first working pipeline
The first end-to-end version of detection, tracking and a minimap, built quickly and judged by eye.