The Little Black Box


#5

Nice job. Would be great to get your source code repository whenever you’re comfortable with sharing it. This is something a lot of people would be interested in. Are you using ssh to push the images to a public server as well for viewing by the public?

This is cool.

BTW, nice shirt.


#6

Thanks. Any thought of production is a ways off as I still have a lot of code to write on the LBB to Theta S control side. I am hoping to implement the full Theta S API through Python scripts. I would like to have the capability to easily configure the Theta S through the use of templates via a webpage.

Are you ready to order?

Bob


#7

Craig,

Yes, it is a nice shirt. I really appreciate my shirt and only wear it for special occasions.

The source code is a long way from being completed but I will put together a block diagram about how all the pieces fit together after I watch the Apple WWDC keynote this morning. A quick answer is that the images are first stored on the Pi Zero in a public folder and viewed through it’s Apache web server. Images can then later be pulled via Samba or pushed via a python script and sftp to a long term storage server.

Bob


#8

Little Black Box Structure


#9

I was taking a little breaking from coding on the little black box today and was playing with Affinity Photo which is my latest iOS app purchase. I have the app installed on an iPad Pro 12.9 generation one. I wanted to see how easy it was to change the starting center of image on a 360 image captured with the Theta S and it was also a good time to test the Little Black Box Samba function. So I got on my MacBook Pro and connected the finder to the Little Black Box via my home network and was greeted with this screen.

Selecting the ThetaS-Pics subdirectory produced this:

I then selected a few photos (17 to be exact) and pasted them to a Theta-S subdirectory I have set up on iCloud.

After the transfer I pulled up the Affinity Photo app on my iPad Pro and added one of my iCloud images using the Affinity Photo Import menu.

Here is the center of picture for the original image:

Moving the center of image and removing the Little Black Box from the photo was a snap.

First select Equirectangular Projection from the Projections menu.

Then find your new center of picture and select Centre Coords.

That’s it for setting the new center. Now for removal of the Little Black Box footprint.

Select Inpainting Brush from the brush menu.

Paint out the area to hide.

And you are done!

Another nice feature of Affinity Photo app is the ability to display image metadata.

The Little Black Box moves pictures out of the Theta S camera storage area as soon as the pictures have finished being processed. The images are then immediately available from the LBB storage area for download. I envision that during a long Time-Lapse runs I will be checking images from time to time and with the easy availability of image metadata, I will be making realtime decisions on what camera parameters need to be tweaked during the run.

Now I have something else to code. Implementing realtime camera changes during the course of a run.

I want to send out a special thanks to @Svendus for pointing out the available of the Affinity Photo app for the iPad while it was still on introductory sale. I think it still might be on sale for anyone else that is interested. It does require an iPad Air 2 or newer. I know for a fact that it will not run on the original iPad Air as I tried it even though the product clearly states that an iPad Air 2 or newer is required to run the app. I just had to try.


#10

Bob,

Reading through your description and comments, I have a couple questions. They kind of jump around, but I’m curious if you have any feedback.

  • Referring the quote above, what types of camera parameters do you think you would be tweaking in real-time? You’re talking about adjusting ISO settings, shutter speed, etc.? This would mostly be for adjusting to darker light conditions?
  • I’m a little surprised that the “balast” for the base is actually components. I guess you have no need for adding more weight? Like a chunk of lead or a roll of coins or something?
  • In your time-lapse submission, you strapped your camera rig onto a fence with bungie cords. Do you envision a square box as a better type of base? (I guess it totally depends on use-case scenarios. Where do you think you’ll be installing the LBB?)
  • I’m really enamoring with the two network solution. In fact, I think it’s essential. It’s really as easy as adding a USB wifi dongle and editing network priority settings? Was there anything else involved in setting that up?

In this run I am using internal power for the Theta S and for the LBB PI but the LBB is capable of having external power (5V micro USB) applied to it as well. When the LBB is receiving external power it can also power Theta S from a USB connector on the side of the box. The LBB charging and boost circuit and associated LiPo battery will ensure that the 5v applied to the Theta S is a clean and constant 5v.

  • I just found out power over USB and date over USB are different things. This is new to me. I guess it’s not much more of an issue than making sure you have the right cable. For a different device (Renesas S3A7 Fast IoT Prototyping Kit) I’m using an external portable battery (2200 mAh capacity, 5V / 800 mA DC input and output). Cool for temporary solutions:

Thanks for any help,
Jesse


#11

Jesse,

Exactly. Modify camera settings on the run to match conditions. And since the LBB let’s you look at the output anytime during a run you can have a good idea what you need to modify. I have also just added the ability to run exiftools on the pictures residing on the Pi. Here is an example (parts are cut off but the text area is a scrollable box).

The weight as is seems fine for the stand and extension that I am currently using.

When I started doing the time-lapse work I was just controlling the camera with my iPhone and running pythton scripts on the phone to control the camera and store the images. This works but it ties up the phone and limits the distance between the camera and phone due to low wifi power and poor antennas. I then moved on to adding a Pi zero W to control the camera and then a second wifi circuit to talk to the Pi zero W via SSH so that I could modify scripts on fly and move pictures off the pi at will. Here is a pictures of that setup. Just add power, selfie stick and bungee cords.

Between the two systems I have lots a flexibility now. Of course all the upgrades that I have done system/software wise can be applied to both hardware configurations.

Minimum you need to set up the Pi to auto connect to the Theta S on one of the wireless networks and you also need to have a wireless access point set up on the second wireless network. The access point is where you can log in (with multiple devices) to do things on the Pi. At a minimum you need to have SSH turned on so that you can use the Pi in a headless mode. With the LBB I have taken everything to the next step by adding a web server, PHP, file manager (Samba), Google VRserver, WIFI Command, and now ExifTools. Here is an updated map of what the LBB can do.

As for power, I believe the internal battery of the Theta S and the LBB can power the system for about seven hours. For really long runs I just add an external 5V power source. A bigger battery or a power adapter. One thing nice about the LBB power system is that it can take a “dirty” input and clean it up to keep the Theta S and Pi happy. The Adafruit charging/boost circuit and the LiPo battery built into the LBB do a really nice job of ensuring a nice clean and constant 5.11V output with no spikes.

I think that I answered all your questions. If not, fire away.

Bob


#12

Getting all the metadata info on the Last Picture image should be really useful. I think people really care about initial orientation, and probably some other details. Do you have any problems with horizon correction? I’m guessing not, cause this is “all THETA.” But pointing out this post on horizon correction just in case.

:slight_smile: It’s cool seeing the “MVP” package as well. :slight_smile:

Also, thanks very much for the LBB components map. Very useful for understanding what you’re doing with your setup.

Also-also, I’m hoping to use your time lapse setup for doing some work in 360 astronomy.


#13

I took a quick glance at the 360 astronomy link. I will have to revisit it when I have a bit more free time. Thanks for the link.

I will eventually give the LBB menu system the ability to get full metadata on any image stored on the Pi’s memory card. It’s on the to-do list.

Let me know if you need any help with the time lapse setup while using my code. There were a lot of parameters that were hardcoded in that Python code.

The LBB will have all the camera option settings available from HTML pages soon. I am currently working on the retrieval of all API options from the Theta S for display via the LBB. The next step will be the providing the ability to change any of the parameters on the camera from the LBB via HTML pages. The last phase of my settings work will be providing the ability of building and storing Theta S setup templates for easy retrieval from pi memory and for application back to the Theta S.

I envision the templates as being a system to take a snapshot of how the camera is setup at a particular time so that you can apply the same settings as a baseline for another shoot.


#14

First Offsite Test of the Little Black Box

Thanks to @Bob_White for allowing me to test the Little Black Box (LBB), “across state lines,” as he says. As a tester, to get started it was easy; I pointed my browser at the webserver housed in the LBB. Bob had temporarily opened up the firewall to all web connections.

After 10-15 seconds negotiating the initial connection, up came the LBB interface, with multiple pulldown menus and buttons.

In general, the responsiveness was very good. I noticed latency just once with the first connection, and then if you took multiple pictures, there is 5 seconds between pictures (the time it takes to stitch two hemispheres together on the camera) plus several seconds before the last pictures shows up in the LBB interface.

The pulldown menu for actions:

At first I tried one picture at high res. I can then choose that picture (or others already in memory on the camera) to see the full metadata. You can view a thumbnail (actually it’s the full image just displayed small) of the equirectangular version. AND Bob has implemented GoogleVR so you can easily see the full navigate-able 360 degree photosphere in your browser, too. Very cool!

The Index page shows the full list of images, in reverse chronological order. Note the bottom 7 are taken today (June 13, 2017) and include 1 high res image (highlighted in red rectangle) and 6 low res tests. Equirectangular versions are on the left, 360 photospheres on the right.

Through the LBB interface, you can run multiple API commands. Here I am running /osc/infoand getting back the current state of the camera:

Finishing up just required turning off the wifi which kills the outside connection.

Comment on power: With smaller electronic gadgets and IoT devices, it’s maybe easy to overlook power. The LBB can take power (cord or battery) to the Raspberry Pi Zero W and power the THETA. Only one power source needed. “It’s super clean,” says Bob.

Conclusion

If you are interested in setting up remote time lapse solutions with a RICOH THETA, you should strongly consider contacting Bob’s consulting firm. He has LBB feature-rich example already built. And his consulting business is open to taking on new challenges and adding different functionality.


#15

@jcasman Thanks for all the help today testing the use of the LIttle Black Box via the internet.

Here is a new map of LBB services adding the remote via internet option.


#16

Bob, thanks a ton. Just posting quickly info about the app you used to map out visually all the services of the LBB. It’s called “iThoughts” by a small UK company called Toketa.

Toketa is a term used in Judo to describe when a person has escaped from being pinned down i.e. they’ve broken free. Seemed apt somehow - software that sets you free. That’s the goal anyhow!

Cool stuff.


#17

When you use the Ricoh Desktop application for horizon correction, doesn’t it just zero out the meta data for camera orientation?

Prior to correction (original image)

After correction with Ricoh Desktop application

I think it zeroes out the image orientation data. I don’t think it corrects the actual image.

In the screenshot below, the top image is the corrected image and the bottom is the original.

The zero out is useful is you edit the image in an image editing application.


#18

The LBB has a second webpage under development.
Right now it will read the Theta S and set pickers to the current settings in the camera.
The next step will be to read changes made to the pickers on this webpage and to write these changes back to the Theta S.

General Information Only Options and Special Options can be viewed from the Tools Menu of the Control Panel. This data is currently displayed as raw data from the Camera. I will be cleaning up this output and also moving some of these options over to the settings menu. These options required slightly different code to extract them from the Theta S hence my orginally grouping them seperate from the main group of options.


#19

Have you considered trying to sell or lease the system to a used car chain? This company uses the THETA.

You’ll need a way to create the virtual tour.

Another market is real estate.

In my opinion, the key is to start off with a vertical industry like auto sales, real estate sales, or surveillance and then make the process of taking images and then displaying them as easy as possible.


#20

Another thing to consider is the upcoming THETA Developer Contest run by the RICOH HQ team. It’s a global contest. The winners of these contests often start their own business.

This was last year’s winners:

These are the winners from 2015

http://contest.theta360.com/2015/result/index-en.html

You can see that HoloBuilder converted this into a business.

The prestige and validation of winning the prize is probably more important than the financial award. Though, the financial award could be used to kick-start the business.


#21

Thanks for the suggestions Craig. I guess that I am far enought along with the product now that I need to start considering opertunties like those.


#22

Again Craig thanks. I am very interested in the next THETA Developer Contest.

Bob


#23

Theta S perched high while being controlled by a Little Black Box system.


#24

Wow, that looks high, easily 15 feet above the rooftop. No weather enclosure needed for the THETA?