Kieran Farr put together a great start at streaming for the THETA S. This is a great way to learn how the conversion works.
For production use, you may want to look at the UVC Blender live streaming driver that RICOH released for Mac and Windows. Download the RICOH driver here.
An experiment to use OS X Quartz Composer to transform Ricoh Theta S live feed dual fisheye to equirectangular.
Goal is to create a Quartz composition that can be used in camtwist or other live video streaming software to adjust live the output of the theta-s to be suitable for using in a 360’ web panorama plugin. The flow will be something like this:
- Theta S via USB or HDMI > Computer
- Computer to use this Quartz composition from within Camtwist > Media encoder > HLS (via cloud encoding service) > web front-end
- Web front-end needs equirectangular image, see this example: https://aframe.io/examples/showcase/sky/
Current state - Proof of concept works
theta-fish2sphere.qtz - this “routes” camtwist through Paul Bourke’s custom quartz patch project (see references below)
@kosso modified version added : theta-fish2sphere_K.qtz with reference images to play with.
Reference Input Image
![‘Reference Input Image’](https://github.com/kosso/theta-s-quartz/blob/master/THETAS_DUALFISH_REF_1280_720.png?raw=true =600x)
Test Output Snapshot
![‘Example Output Image … so far’](https://github.com/kosso/theta-s-quartz/blob/master/OUTPUT.png?raw=true =600x)
- Sample input file (raw dual fisheye from theta-s): https://github.com/kfarr/theta-s-quartz/blob/master/theta-s-raw-Frame-01-03-2016-02-00-34.jpg
- Example of desired output (this is post-processed by theta-s software, but not available for live streaming): https://github.com/kfarr/theta-s-quartz/blob/master/desired-output-R0010093.jpg
- Download camtwist http://camtwiststudio.com/, install to applications
- Add theta-fish2sphere.qtz and theta-crop.qtz to Applications > CamTwist > Effects
- Open CamTwist
- Step 1: View > Main Window, Double click “Webcam” and choose RICOH THETA S
- Step 2: Choose filter theta-fish2sphere
- Step 3: Modify settings if you’d like, not sure it’ll make much of a difference yet. Make sure to press enter after you enter new values.
How to modify:
- Download quartz composer from Apple’s developer site http://developer.apple.com/downloads by searching for “Graphics Tools for Xcode”
- Open theta-fish2sphere.qtz file in quartz composer
- Change input image source to theta s on USB or HDMI input live stream, or use sample image from this git project’s root dir
Dev notes for Quartz Composer
- For testing / developing in QC, you’ll need to unhook the “Image (Required)” patch and use instead the image patch “image-example of 720p theta-s stream”
- Ideally in QC during dev one could use video input patch to use as reference for modifying settings, but it is not consistent with behavior when ran inside of camtwist
Some helpful docs
- Paul Bourke - Converting a fisheye image into a panoramic or perspective projection - http://paulbourke.net/dome/fish2/
- Another attempt using opencv instead of quartz: https://github.com/kscottz/dewarp/blob/master/fisheye/defish.py
- Another attempt use processing.org PDE: https://github.com/theta360developers/THETA-S-LiveViewer-P5
- Another nice blog post: http://www.kscottz.com/fish-eye-lens-dewarping-and-panorama-stiching/
- Theta-S camera: https://theta360.com/en/about/theta/s.html
It is theoretically possible to experiment with the HDMI output of the Ricoh Theta S > Cable (Mini HDMI Output to Standard HDMI) > Capture Device (Blackmagic UltraStudio Mini HDMI Input to DisplayPort) > Quartz Custom Composition* > Quartz Syphon Plugin Output > OBS Syphon Receiver > RTMP > Streaming Service
*Quartz Custom Composition is the remaining step not complete. The current quartz composition in my git repo is only made for CamTwist and oriented for 720p not 1080p. It would be possible to adjust the Quartz Composition through (a) wise thinking with maths or (b) blind guessing and luck tweaking variables. (b) is my approach and it got us surprisingly close for 720p (UVC blender is still better).
Also caveat that *Quartz Custom Composition may require processing power above the capabilities of a standard MacBook Pro.