Skip to main content
Loading...

Stitching 360 panorama with Raspberry Pi CM3, StereoPi and two fisheye cameras. Step-by-step guide.

  • 360 panorama with StereoPi

In this article we will continue our experiments with the StereoPi stereoscopic camera based on the Raspberry Pi Compute Module. This time, we will create a 360 degree panoramic photo!

Click on image for online panorama view

Intro

In our last experiments, we installed cameras side-by-side with parallel axes and worked with stereoscopic effect. Today, we will use an inverted approach: cameras pointed in opposite directions, but equipped with wide-angle fisheye optics, each with a 200 degree field of view.

Let’s start at the end: here’s our resulting creation — basically, a panorama.

Hardware

We have a StereoPi board

StereoPi board

…with Raspberry Pi Compute Module 3 Lite inside…

RaspberryPi Compute Module 3 Lite

…and two wide-angle RPi (M) WaveShare cameras:

RPi Waveshare cameras

We then attach the cameras back-to-back:

StereoPi cameras back-to-back, 360 degree setup

 

Then, we capture a photo with each camera, and get these two pictures using raspistill:

200 degree fisheye photo 1

Full size version

 

200 degree fisheye photo 2

Full size version

Panorama stitching

Here, a question arose: how could we combine these two images into one with an equirectangular projection, such as is supported by almost all panorama viewing software?

After a long investigation, we found a 360-camera project, which was used as a reference for our future code.

However, to start stitching, we would have to start with a desktop panoramic application, to prepare a template for future transformations. For this, we used Hugin, which is open-source and can be downloaded here: http://hugin.sourceforge.net/download/

1. So, we’ve downloaded, set up, and started up the software.

2. Now, we need to choose “Simple” in the “Interface” menu.

StereoPi Hugin simple interface

 

3. Press “Load images…” and add two our files (21.jpg и 21–2.jpg).

StereoPi Hugin load images

 

4. Set “Lens type” to “Circular Fisheye”. “Focal Length” should be set to 1,2 mm, and “Focal length multiplier” to 7,6x. In the “Projection” tab, check that “Field of view” is set to 360x180 and “Equirectangular”. These are the default settings.

StereoPi Hugin load images 2

 

5. Next, press the button “2. Align…”. This begins the search for control points, which should find around 10–13 points. At this step, our panorama is already starting to look like a panorama.

StereoPi Hugin equirectangular preview

 

6. And now for the most important step — saving the project for later use in the automatic stitching of all consecutive panoramas captured by our 2 fisheye cameras. To save, go to“File” -> “Save as…” -> filename “stereopi-template.pto”

7. Then, we go back to the “Assistant” tab (if we happened to go out of it) and press “3. Create panorama”. A new window appears, in which we set a height of 1944, to which the width adjusts automatically. The LDR format (resulting picture format) should then be set to JPEG. The image quality is at your discretion; the default for this is 90. This time around, we don’t tinker with“Corrections” and instead press “Save”.

StereoPi Hugin saving settings

 

8. After this, several windows appear, one of which will show the progress log.

StereoPi Hugin status window

 

9. In the end, we get something that looks like this:

StereoPi equirectangular image

Full size version

This is exactly what we needed- an equirectangular projection of our spherical panorama!

Porting stitching onboard

Ok, so all of this is great- but do we really want to bother with each captured photo?

To up our efficiency, we need to make this process automatic. For this, we will need to use our project file, saved at step 6. We take this file (in our case it’s called stereopi-template.pto) and copy it over to StereoPi.

We also copy our script, stereopi-stich.sh. This script needs us to input parameters — namely, the file names of the fisheye photos to be stitched to equirectangular projection.

But first, we need to install all the required software for StereoPi. Simply download this script on StereoPi and run it: installer.sh

Now let’s run the stitching script:

# ./stereopi-stich.sh 21.jpg 21–2.jpg 
Stiching files 21.jpg and 21–2.jpg 
Generating pto file… 
Reading /opt/Pano/test1/21–2.jpg… 
Reading /opt/Pano/test1/21.jpg… 

Assigned 1 lenses. 

Written output to /opt/Pano/test1/tmp/project.pto 

Written output to ./tmp/project.pto 
number of cmdline args: 1 
================================== 
Stitching panorama 
================================== 
nona -z LZW -r ldr -m TIFF_m -o 21_21–2-pano -i 0 ./tmp/project.pto
nona -z LZW -r ldr -m TIFF_m -o 21_21–2-pano -i 1 ./tmp/project.pto
checkpto — generate-argfile= project.pto_21_21–2-pano.arg
./tmp/project.pto 
enblend — compression=90 -w -f2688x1344 -o 21_21–2-pano.jpg — 21_21–2-pano0000.tif 21_21–2-pano0001.tif 
enblend: info: loading next image: 21_21–2-pano0000.tif 1/1 
enblend: info: loading next image: 21_21–2-pano0001.tif 1/1 
enblend: info: writing final output 
Bogus input colorspace 
exiftool -overwrite_original_in_place -TagsFromFile
/opt/Pano/test1/21–2.jpg -WhitePoint -ColorSpace -@ /usr/share/hugin/data/hugin_exiftool_copy.arg -@ project.pto_21_21–
2-pano.arg 21_21–2-pano.jpg 
1 image files updated 
================================== 
Remove temporary files 
================================== 
rm project.pto_21_21–2-pano.arg 21_21–2-pano0000.tif 21_21–2-
pano0001.tif


The process takes about 50 seconds. It’d be good to look for some optimizations for it in the future, but right now it’s good enough for testing purposes

This file is the result:

StereoPi Hugin equirectangular image transformation result

Full size version

Voila! We’ve automatically stitched two fisheye images into one equirectangular file! Now, we can use this script for stitching all future files. The key point is to avoid changing relative camera positions — otherwise, the stitching will be of lower quality.

Now, we just need to insert it somewhere on our web-page. A panorama player is required to view the panorama..

We think the best embedding player for panoramas is KRPano. It supports both photos and videos. However, a paid license is required for usage. We bought a license about 5 years ago, but at that time it was based on Adobe Flash, and didn’t support HTML5. Now it can, so we plan to buy new license. https://krpano.com/

You can download my archive with all you need to obtain this result. The funniest view mode is “Little planet” (to change the view mode, just click the right mouse button).

StereoPi little planet 360 photo effect

 


We hope that our experiment will not only be interesting for you, but will also be useful as a step-by-step manual. Thank you for your attention!


Useful links: