# A universal ambilight

**URL:** <https://discourse.processing.org/t/a-universal-ambilight/1962>\
**Category:** Gallery\
**Created:** [July 21, 2018, 4:28pm UTC](https://discourse.processing.org/t/a-universal-ambilight/1962 "2018-07-21T16:28:06Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![jb4x](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.processing.org/jb4x/32/789_2.png) [@jb4x](https://discourse.processing.org/u/jb4x)\
**Post date:** [July 21, 2018, 4:28pm UTC](https://discourse.processing.org/t/a-universal-ambilight/1962/1 "2018-07-21T16:28:06Z")

</div>

Hi everyone!

This project is still a work in progress but I would like to share with you the progress I have made so far, the direction I want to follow and gather some feedback or advice about it.

I wanted to build an ambilight system for my TV but the problem I ran into was that I was and still am using only an ethernet cable to get my video feed. So I couldn’t use those solutions you can find online using the HDMI, USB or VGA feed.

For those who don’t know what an ambilight system is, here is a link to a corresponding wikipedia page:

> **[Bias lighting](https://en.wikipedia.org/wiki/Ambilight)**
>
> In home cinema and video editing technology, bias lighting is a weak light source on the backside of a screen or monitor that illuminates the wall or surface behind and just around the display. 
> The purpose of bias lighting is to reduce the perceived brightness of the display as a result of the contrast with the slightly illuminated area around it. This reduces the eye strain and fatigue that occurs when viewing a bright display against a very dark background for an extended time, and increases ...

To overcome that problem, I got an idea to obtain that feed through a camera and use that as my source to detect the colors of my LEDs.  
For this, I will connect a camera (RPI04V2) to a Raspberry PI 3 B+ running a processing sketch. That sketch will have the role to read the feed form the camera and figure out the color of the LEDs for the current image. Then those information will be sent by bluetooth to an arduino connected to the LED.

Now, I am not completely sure it will work because of 3 main factors :

- the delay that will be introduced between the moment the image is displayed and the moment the LEDs get the information. I have done some tests already and I think it will be ok for normal scenes but maybe a bit slow for action scenes.
- The quality of the feed : because it is a camera, some darker and brighter zones appear on the image and it can affect the color of the LED. I have already thought about some ways how to tackle this issue and ran some positive tests but it still needs a bit more work
- The camera parameters : I’m afraid that when the image becomes brighter the camera will adapt its sensitivity to show a darker picture and the other way around for a dark image. I haven’t worked on that one yet.

You can find the current version in this github repository:

> **[jb4xx/AmbiCam](https://github.com/jb4xx/AmbiCam)**
>
> Contribute to jb4xx/AmbiCam development by creating an account on GitHub.

What is already done:

- Display the list of the available cameras and select the one of interest
- Detect the TV screen position by displaying a unique color on the TV screen
- Straighten the image of the TV

What needs to be done:

- Apply the color correction to the image
- Make it work on the raspberry PI
- Get the LEDs color
- Send the information to the arduino via bluetooth
- Build the LEDs panel
- Write the arduino code

If you have any idea on how to improve it or if you want me to explain in a bit more in detail how some algorithms work, feel free to post a new message 🙂

---

<div class="post-metadata">

**Author:** ![jb4x](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.processing.org/jb4x/32/789_2.png) [@jb4x](https://discourse.processing.org/u/jb4x)\
**Post date:** [August 18, 2018, 8:26pm UTC](https://discourse.processing.org/t/a-universal-ambilight/1962/2 "2018-08-18T20:26:29Z")

</div>

Hi everyone,

I finally got some time and courage to continue this project.

I started developing the concept on my windows computer using the video library provided by the processing foundation. Sadly, I figure out the hard way that I can’t use that library with a Pi board so I used the [glvideo library by gohai](https://github.com/gohai/processing-glvideo) instead. I faced many problems to transcript the previous code to the new one and I ended up with big changes in the structure. All that to say that I have created a new repository on github to get a fresh start (the old one is still available):

> **[jb4xx/AmbiCam\_V2](https://github.com/jb4xx/AmbiCam_V2)**
>
> Contribute to jb4xx/AmbiCam\_V2 development by creating an account on GitHub.

Now let’s get to the fun part, some screenshots of how it works.

For testing reasons, the tv is replaced by a piece of paper sticked to the wall:

 ![Testing setup](https://canada1.discourse-cdn.com/flex036/uploads/processingfoundation1/original/2X/7/7bbfebda797cd2474a56a93015f0020f08ed35eb.jpeg)

First, you start with a menu giving you all the shortkeys that can be used:

 ![Tool menu](https://canada1.discourse-cdn.com/flex036/uploads/processingfoundation1/original/2X/1/108ee4c557155788bfa4a917f0909f249e52f0a0.jpeg)

Then you need to calibrate the tool. It consists of 2 steps:

1. Display a unique color (different enough than the background) on your tv
2. Change the threshold value until only the screen is selected  
The tool assists you by displaying a red overlay on top of the pic, showing the selected area.  
You can see the difference between a bad threshold and a good one on the 2 followings screenshots:

 ![Bad threshold](https://canada1.discourse-cdn.com/flex036/uploads/processingfoundation1/original/2X/f/f5b103bc71dde07f359056f98e9ee83ef92791fd.jpeg)

 ![Good threshold](https://canada1.discourse-cdn.com/flex036/uploads/processingfoundation1/original/2X/7/7b4c32210c8049f873cc05726459a491e8f5bc3f.jpeg)

Once you found a good threshold, the tool bakes some calculations needed at every frame for better performances.  
The first task is to find the corners of that area. The second debug mode can show that this operation has been successful:

 ![Corner detection](https://canada1.discourse-cdn.com/flex036/uploads/processingfoundation1/original/2X/a/aa036d449a9250574fea6dfa4c12f09cfa8a8c7e.jpeg)

The second task is to reshape that tetragon to make it a perfect rectangle. In order to save some performance, the accuracy of the process can be changed and only the outer edges (that we are really interested in) is straightened.  
To better show the result the background pic is changed:

 ![New background](https://canada1.discourse-cdn.com/flex036/uploads/processingfoundation1/original/2X/f/f103e28473606ce7e6d7e8741a3aaf717901ac2b.jpeg)

 ![Reshaped screen](https://canada1.discourse-cdn.com/flex036/uploads/processingfoundation1/original/2X/e/ed23837341c2f8819bee8d9e082afa9216852bbc.jpeg)

After, it is just a matter of figuring out the proper color to display. Here are the result with the previous background and the result with a new one to better see the effect:

 ![Result 1](https://canada1.discourse-cdn.com/flex036/uploads/processingfoundation1/original/2X/8/831824b3c67aba5459e07db5ee2e4063dcf7af88.jpeg)

 ![New background](https://canada1.discourse-cdn.com/flex036/uploads/processingfoundation1/original/2X/e/eb7d1bd6fde63f62c3a33c0a27c519bb84a55aa6.jpeg)

 ![Result 2](https://canada1.discourse-cdn.com/flex036/uploads/processingfoundation1/original/2X/0/0b7ce1037bbb70a0313820e94e9cebd69ce9465f.jpeg)

I’m quite happy with the progress but there is still plenty of work to do:

1. Now that I have the result that I wanted I need to figure out a way to send it via bluetooth to an arduino board
2. Then I would need to build the LEDs structure and make it works for real!
3. Keep improving:
  - Change the way the colors of the LEDs are computed. Right now it is just a simple resize function but I’m not sure it gives the best result
  - Compensate for brightness difference over the screen. I already have a working exemple but I think it can be improved even more.
  - Provide a way to tweak the colors of the LEDs without changing the output of the sketch.
  - …

If you have any question or advice please go ahead, and feel free to use it, modify it, improve it… 😃

I also want to thank @gohai, @msurguy, @GoToLoop and @neilcsmith who help me find my way around using a camera on a Pi.

---

<div class="post-metadata">

**Author:** ![gohai](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.processing.org/gohai/32/25_2.png) [@gohai](https://discourse.processing.org/u/gohai)\
**Post date:** [August 19, 2018, 8:10pm UTC](https://discourse.processing.org/t/a-universal-ambilight/1962/3 "2018-08-19T20:10:35Z")

</div>

Cool project - thanks for sharing the detailed write up @jb4x!

---

<div class="post-metadata">

**Author:** ![jb4x](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.processing.org/jb4x/32/789_2.png) [@jb4x](https://discourse.processing.org/u/jb4x)\
**Post date:** [September 15, 2018, 6:01pm UTC](https://discourse.processing.org/t/a-universal-ambilight/1962/4 "2018-09-15T18:01:51Z")

</div>

Hi everyone,

Just a quick update, the project is not dead.

I finally found some times to complete the frame:

 ![Frame_01](https://canada1.discourse-cdn.com/flex036/uploads/processingfoundation1/original/2X/b/bafddbb95b8faf9ded00f5d1a865bbca9601d811.jpeg)

![Frame_02](https://canada1.discourse-cdn.com/flex036/uploads/processingfoundation1/original/2X/1/14e4f53b6ab996cd47004dbf3b8e4be76d61001e.jpeg)

I also had some problem pairing the blueduino rev3 to my raspberry 🤪

It should be all good now.

The only thing left is the code to send the data to the blueduino and the bueduino code.

---

<div class="post-metadata">

**Author:** ![jb4x](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.processing.org/jb4x/32/789_2.png) [@jb4x](https://discourse.processing.org/u/jb4x)\
**Post date:** [September 27, 2018, 7:10pm UTC](https://discourse.processing.org/t/a-universal-ambilight/1962/5 "2018-09-27T19:10:10Z")

</div>

Alright guys,

Today’s a really exciting day 😃😃🤩

I finally got it to work!

Here’s a video of how it performed:

[![](https://img.youtube.com/vi/gUWPgcX7Ehc/maxresdefault.jpg "Universal ambilight test 01") ](https://www.youtube.com/watch?v=gUWPgcX7Ehc)

There are few points to improve though:

- The lights are flickering because of the camera feed
- The colors are not always completely right
- It is not wireless (I used a usb cable) - Not sure if bluetooth will be fast enough though
- There is a little delay between the LEDs and the TV

---

<div class="post-metadata">

**Author:** ![jeremydouglass](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.processing.org/jeremydouglass/32/20_2.png) [@jeremydouglass](https://discourse.processing.org/u/jeremydouglass)\
**Post date:** [September 28, 2018, 8:12pm UTC](https://discourse.processing.org/t/a-universal-ambilight/1962/6 "2018-09-28T20:12:54Z")

</div>

Very cool demo!

In addition to flicker reduction, you might also try out a smoothing / crossfade operation with an adjustable duration – although if I understand right the original aesthetic of the ambilight is to be instantaneous.

---

<div class="post-metadata">

**Author:** ![jb4x](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.processing.org/jb4x/32/789_2.png) [@jb4x](https://discourse.processing.org/u/jb4x)\
**Post date:** [September 29, 2018, 6:36am UTC](https://discourse.processing.org/t/a-universal-ambilight/1962/7 "2018-09-29T06:36:59Z")

</div>

Thank you @jeremydouglass for the feedback, really appreciated! 🙂

The problem I have with flicker reduction is that I think it is hardware related. I had huge trouble making it works and currently, for example, I can’t use the camera at more than 20 fps when I should, in theory, be able to go up to 60 fps…

Another issue I have with the camera is the auto-brightness adjustment: I have no control over it what so ever and it is not a simple light sensor outside the camera that I can tape 😕

And I don’t think I can do something about that flickering effect by code.

Now for the smoothing/crossfade operation, as you said, the goal would be to be as instantaneous as possible to make it feel like it is an extension of the screen. The smoothing would break that effect.

Now if push completely to the limit, the idea can be to not want a perfect match with the screen but to have more like an ambiant lighting around the screen. It would not use just the pixels on the edge but the whole screen, sense the ambiance of the scene, and display something accordingly.

If people have ideas linked to that, I would be happy to read them 🙂

---

<div class="post-metadata">

**Author:** ![jeremydouglass](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.processing.org/jeremydouglass/32/20_2.png) [@jeremydouglass](https://discourse.processing.org/u/jeremydouglass)\
**Post date:** [December 20, 2018, 5:28pm UTC](https://discourse.processing.org/t/a-universal-ambilight/1962/8 "2018-12-20T17:28:40Z")

</div>

I just linked this recent discussion of the Adafruit Adalight, which seems related to your project –

> [@Annotation processing got disabled, since it requires a 1.6 compliant JVM](https://discourse.processing.org/t/annotation-processing-got-disabled-since-it-requires-a-1-6-compliant-jvm/6697/4):
>
> Thanks to the both of you for you replies and sorry for the delay. I am new to processing and I have seen both of the posts that you mention in my research prior to posting my question but as I am new to processing I did not think that they applied. As I write this I am still confused and do not know if there is a solution for my issue. Background: I am trying to build up an Adalight setup and I have gotten it running on a Windows 7 machine as well as a Linux Mint machine however when I go to…

> **[Adalight - DIY Ambient Monitor Lighting Project Pack](https://www.adafruit.com/product/461)**
>
> Build your own ambient-light addition for a monitor or media PC television with the Adalight project pack! This project pack is for our "Adalight" project tutorial. By running the ...