Case study of our interactive projection mapping at Kitamido for Midosuji Illumination 2026
For Midosuji Illumination 2026 in Osaka, Utsubo created the 15-minute interactive content and the software for the projection mapping at Kitamido (Hongwanji Tsumura Betsuin). Sensors follow visitors across the temple forecourt, and the floor, stairs and façade respond to them. PixelEngine, our client on the project, supplied and installed the projection equipment.
Who this is for: municipalities, event organizers, venue owners and production companies planning a projection mapping that people can walk into, not only watch.
Key Takeaways
- The goal: push how three-dimensional a projection mapping can look, and make it react to the people standing in it.
- The scale: 15 minutes of content on the temple façade, stairs and forecourt, run through 10 projectors and tracked by 4 LiDAR sensors.
- The software: we built the show on our own node-based real-time engine, so a small team could test on a 3D model of the temple and change any scene right up to opening.
- The result: on opening night, about 13 people were interacting on the forecourt at any moment, with a peak of 34, and a larger crowd watched around them. A live TV report tried it on air.
- The main lesson: motion that looks right on a monitor is too fast on a building. Many scenes ended up at half speed or slower after on-site review.
1. Project Overview
Kitamido's interactive digital art is part of Midosuji Illumination 2026 and runs at the temple forecourt until December 31.
| Item | Details |
|---|---|
| Venue | Kitamido (Hongwanji Tsumura Betsuin), Osaka |
| Site size | Forecourt about 55 × 15 m. Main stairs 18.6 m wide, with 45 steps climbing about 7 m (measured on the 3D model) |
| Event | Midosuji Illumination 2026, part of the Festival of the Lights in Osaka |
| Dates | November 3 to December 31, 2026 (no show November 17–24 and December 11) |
| Hours | From about 17:00 to 23:00 |
| Client | PixelEngine Inc., Kyoto |
| Utsubo's scope | Concept and art direction, 15-minute content, interactive system including the LiDAR tracking logic, output integration, LiDAR calibration and fine-tuning in our software, content adjustment on site, remote maintenance |
| PixelEngine's scope | Supply and setup of the projectors, LiDAR sensors, playback PC and sound equipment, projector alignment, soundtrack, 3D scan of the site |
| Equipment | 10 projectors, 4 LiDAR sensors, 2 × 4K video outputs |
Dates, hours and closures come from the festival's official Midosuji Illumination page. Check it before you visit.
2. The Brief: More Depth, and a Show That Reacts
The brief asked for an interactive work on the theme of connection, in a venue with no single viewing position. Midosuji turns 90 next year, and the 2026 illumination is themed "LINK".
Two aims shaped everything:
- Make the building look truly three-dimensional. Most mapping paints flat images onto a surface. We wanted light and shadow that behave as if they belong to the real columns, roof and stairs.
- Make visitors part of the show. The forecourt is open, so people walk through the projection instead of watching from a fixed seat.
The venue made this hard. Visitors keep moving, and 45 stair risers break any image that crosses them. The music and the 15-minute length were also fixed early, so the visuals had to fit them. We built the piece as an anthology of different worlds, so each one could work on its own.
3. The Experience: What Happens When You Walk In
Each scene gives the forecourt a different reaction, so people keep discovering new ones.
| Scene | What visitors do |
|---|---|
| Particle rings | A ring of light forms around each person and follows them |
| Flowers and petals | Flowers open where people stand and walk |
| Pink bubbles | Bubbles pop when you step on them |
| Blue blocks | The ground breaks apart under your feet |
| Glitch | The building and floor break into pixel blocks, and markers show arrows pointing where each person is heading, based on their velocity |
| Sea creatures | Fish swim away from people and crabs flip over when you touch them, while a whale crosses the façade |
| Glowing balls | A physics simulation pours balls down from the roof. They pile up on the steps, and you can make them bounce by walking up to them |
There are no instructions. Children usually work it out first, and adults follow. After on-site reviews, we made many reactions larger and clearer, because an effect has to be readable from where the participant stands, not only for people watching from the side.
4. How We Pushed the 3D Effect
The show looks three-dimensional because every effect is calculated on an accurate 3D model of the temple.
Because the effects know the temple's real surfaces, they can use the building itself:
- Virtual lights cast shadows from the real columns, steps and walls.
- Glowing lines run along the actual roof edges, door frames and stair edges.
- Physics follows the architecture, so simulated balls bounce down the real steps and settle on them.
We then checked each scene against the real thing. These are engine previews and the same scenes on opening night:
LiDAR scan plus Gaussian splatting
We combined two captures of the site. The 84-million-point LiDAR scan gives the physically precise geometry that the mapping and the physics simulations rely on. A Gaussian splat, trained from photos we took of the temple from every side, adds the real colors and light for a photorealistic render.
That geometry is what the physics runs on. Here is an early production test of the simulated balls flowing down the temple's 3D model:
5. Our Own Software: Built for This Kind of Show
We ran the project on a node-based real-time engine that we build ourselves, because it let a small team test, branch and fix a live show quickly. Kitamido was the first show to run on it.
The glitch scene at three stages: its nodes, the engine preview and the temple on opening night. These previews come from a production version, when the glitch was still black and white.
| What we needed | What the software did |
|---|---|
| Test before reaching the site | A 3D model of the temple, simulated projectors and simulated visitors, so we could rehearse with no equipment |
| Iterate as a team | Live shared projects and branches. Each person changed a scene on a branch, then merged it into main, which reached version 3,138 by the eve of opening |
| Add what was missing | We shipped 53 engine updates in the month before opening, as the show asked for new features |
| Track people | 4 LiDAR sensors merged into one map of people on the forecourt, fine-tuned in the software to ignore parts of the site. No faces involved |
| Run by itself | A schedule that starts each day, eases the sound down for the last minutes and ends the show before closing |
| Stay stable | A frozen copy of the show for the venue PC. In a 6-show test, GPU memory matched to the megabyte from show to show |
| Support it remotely | We monitor the show's health during the run, so most issues could be solved from our Osaka office |
| Measure it | Frame rate and the number of people on the forecourt, recorded every minute and kept for 14 days. The opening-night numbers below come from this |
6. From First Concept to Opening Night
Production moved from a shared base project to opening in about four weeks.
| Date (2026) | Milestone |
|---|---|
| September 28 | Concepts presented and main direction chosen |
| October 2 | Moving storyboard reviewed with PixelEngine |
| October 5 | Production starts on a shared base project for the whole team |
| October 14 | First 10 minutes of the video reviewed |
| October 20 | First projection test on site |
| October 24–27 | On-site reviews and feedback |
| November 1 | Final changes merged |
| November 3 | Opening night |
7. What We Learned
Most of what we changed on site came down to speed, contrast and clarity.
- Slow everything down. A building is huge, and motion feels much faster on it than on a screen. The whale went to half speed, and some scenes dropped to a third.
- Contrast beats detail. Small graphics disappear at this scale. Cutting background light to near zero made the main subjects stand out far more than adding detail.
- Give people time. Visitors need several seconds to notice that the floor is reacting. A scene that passes too quickly leaves no time to play.
- Test what you can only see at night. Projection is invisible in daylight, so every on-site check depends on the few dark hours available.
- Plan for a long run. A show that repeats every night needs a soak test, monitoring and remote support from day one.
8. Opening Night
On November 3, about 13 people were interacting on the forecourt at any moment across the six-hour evening, with a peak of 34. More people watched from around it, and they are not in these numbers.
- The crowd came with nightfall. The count jumped to 33 at 17:29, just before the 17:30 run, and stayed between 13 and 28 for all 15 minutes.
- The busiest stretch was 17:30 to 19:00. The 18:45 run averaged 22 people, with a peak of 31.
- It stayed busy. The forecourt held 10 or more people for 69% of show minutes, and 20 or more for 52 minutes.
- People played until the end. The last half hour still averaged 8 people, with a peak of 14.
In total, that is 4,508 person-minutes, or about 75 hours of people interacting with the work in one evening.
That evening, Kansai TV, one of the main broadcasters in the Osaka region, covered the illumination live on its news program news Runner. In this clip (in Japanese, from 2:38), the report presents Kitamido as "something that makes you feel the future" and "not just any projection mapping". The reporter then walks onto the forecourt and shows the floor changing with each step: "I've tried it many times today, and it's so much fun I can't stop."
9. Could This Work for Your Venue?
Yes, if the site can be dark at night and has space for people to walk. The same approach works on temples, public buildings, plazas and commercial façades, with equipment from your partner or ours.
- Comparing formats? See interactive installation vs projection mapping.
- Budgeting? See our projection mapping cost guide.
- Working with a place of worship? See our guide to worship space installations.
10. About Utsubo
Utsubo is an interactive studio based in Osaka, founded in 2021, working in English, Japanese and French. Our work includes Waves of Connection at Expo 2025 Osaka, which reached 10,000+ visitors, and projects for JR West and SMRJ. We have won 15+ FWA and Awwwards honors.
11. Let's Talk
Planning a projection mapping or illumination that should react to people? We work with venues, organizers and production companies, either on the full project or on content and software alongside your equipment partner.
If you're exploring a partnership, let's discuss your project:
- Your venue, dates and how visitors will move through it
- Whether interaction fits, and which sensors it would need
- Who handles equipment, and where we fit in
FAQs
What did Utsubo make for Kitamido?
Utsubo made the concept, the 15-minute interactive content and the software that runs it, including the LiDAR tracking logic, and handled LiDAR calibration and fine-tuning in that software (for example, which parts of the site to ignore), content adjustment on site and remote maintenance. PixelEngine supplied and installed the equipment, including the LiDAR hardware, aligned the projectors and provided the soundtrack.
When and where can I see it?
At Kitamido (Hongwanji Tsumura Betsuin) in Osaka, from November 3 to December 31, 2026, from about 17:00 to 23:00. There is no show from November 17 to 24 or on December 11. Check the festival's official page before visiting.
How does the projection react to visitors?
Four LiDAR sensors detect where people are on the forecourt. Our software turns those positions into real-time effects, such as rings that follow people and bubbles that pop underfoot. LiDAR measures position and does not capture faces.
How many projectors does it use?
Ten projectors cover the façade, stairs and forecourt, fed by two 4K video outputs from one PC. PixelEngine supplied and installed them.
Why did you build your own software?
We wanted one tool for testing on a 3D model, working as a team with branches, tracking people and running the show every night. Building it ourselves also let us add features during production when the show needed them.
Can you do this on our building?
Yes, if the site can be dark enough at night and has room for visitors. We can work with your equipment partner or bring in one of ours. A 3D scan of the site is the usual first step.
Osaka Interactive Installation Studio
