making stereoscopic 3D from the Valve Index cameras 

the Valve Index has two front cameras, available as a single video device with dual fisheyes

this means that to do anything with it, you need to figure out the cameras' properties, such as field of view, distortion, etc., figure out the cameras' properties relative to each other, correct the distortions, etc.

OpenCV comes with a lot of nice functions to do that: it has built-in functions to find markers and checkboard patterns in images, functions to figure out cameras' characteristics from sets of images taken with it correlated with those points, things to compute correction transforms…

so far what i've been doing is:
- film a chAruco (checkboard + markers) board (displayed on my computer screen) from different angles from my Valve Index
- use OpenCV functions to find aruco markers on both cameras
- use OpenCV functions to find checkboard corners from those patterns, on both cameras (this gives corner identifiers and their position within the two images)
- check which corners can be seen by both cameras
- from this set, compute the object position (coordinates) of the corners (annoyingly, the charuco functions do not expose anything to get those values, i need to recompute them myself)
- store the set of object positions, left images and right images (tricky bit: corners are not necessarily reported in the same order on both cameras, so care needs to be taken to avoid mismatched objects)
- give all of that to an OpenCV function that computes camera intrinsics and distortion values for both cameras as well as their relative positions
- undistort each camera separately

now i need to use OpenCV's functions to correct the stereo view, rotating, cropping and translating both camera views as needed

Follow

yeah i have spent far too much time on that and i have a headache, but,

making stereoscopic 3D from the Valve Index cameras 

well,
i think i applied all the needed transformations with OpenCV
but it still looks off

Sign in to participate in the conversation
Mastodon (instance perso)

This is a small personal instance running on a couple small ARM servers at home.