My Ambilight, My Obsession, My Curse

Introduction

Like many others, I built a DIY ambilight setup for my TV.  First let me say, I am not a software developer, nor am I hardware developer. If I were, I may not be writing this post. I am merely a hobbyist and I would be absolutely overjoyed if a well-seasoned veteran in either of those areas read this post and had a solution for me. With that out of the way, a disclaimer: this is going to be a very long, rambling, post. I have spent the better part of the last year with my mind subtlety occupied by this subject. I think about it for at least a few minutes every day. This post is going to be a sort of dump of my collective learning on the subject, mostly just as an attempt at getting it out of my brain, and secondarily as a hope that perhaps someone will stumble across it and offer me a solution I may not have considered.  There is no TL;DR, so either read it, or don’t.

For this not familiar with what an Ambilight Clone is, here is mine in action:

Background

About three years ago I, like many others, decided to build an Ambilight clone using a Raspberry Pi. I was quite successful. I can’t help but to feel I was on the early side of this trend but I hate sounding like a hipster, because I certainly didn’t come up with the idea or anything. I mean that only to say that I built my setup 100% from scratch, even to the point of having to heavily modify the original Boblight code to run on Arch Linux ARM with newer versions of FFMPEG, as well as be fast enough to cope with the larger than average number of LED’s I was using. While I won’t go into the deep details of this setup (I may do a retroactive write-up at some point), at a high level it required XBMC / Kodi via the Boblight plugin to feed a daemon on the Raspberry Pi over the network. This meant that my ambilight only worked with content that I could view on my HTPC, specifically inside Kodi. Technically I could have had any graphic on the OS (including the desktop or browser-based video streaming) feed the light, but this wasn’t exactly couch (wife) friendly, so I artificially limited myself to Kodi.

Somewhat recently I decided to switch from Kodi to Plex. There are many reasons for this but they are mostly irrelevant to the conversation. What is relevant is that Plex, out of the box, does not support plugins like Kodi does, specifically the Boblight plugin. To deal with this, I turned my old HTPC into a Plex Media Server, and upgraded my original boblight Raspberry Pi to a Raspberry Pi 3 which would run both the Plex Player, and a local instance of Hyperion. Instead of using my usual Arch Linux build, I went with RasPlex because it has Plex Home Theater and Hyperion support out of the box and I was able to get my existing setup back up and running on Plex very quickly and easily.  Making this change forced me to revisit my long running ambilight setup.  My setup was so reliable that I had not looked into it or kept on top of the technology since the day I first got it working.  There was no need to follow the tech, because it worked.

As it turns out, getting my light setup working with Plex was not the end, but only the beginning of my journey.  I had just re-opened the Pandora’s box of my OCD mind. During the move over to Plex/Hyperion I realized a couple of things: One, I am very attached to my Ambilight, to the point where the idea of just getting rid of it never entered my mind. Two, I was tired of being limited as to the content which could drive the lights. While it is mostly fine that only the content I download works, as time moves forward I fear that the amount of content that falls into this category will be less and less.  I find myself using services like Netflix, Amazon, and On Demand more and more all the time.  If the industry continues advancing and provides me with reasonable, cost-effective ways to get content, I am happy to pay for it.  If Netflix and Amazon gets movies the day they are released on Blu-ray available to stream without an additional rental fee, I would never download another movie. Three, in figuring out how to re-implement my light setup with Plex, I discovered a whole new population of people who had implemented an ambilight, as well as a lot of new software implementations I could play with.

Status Quo

One of the drivers of moving from Kodi to Plex was that I used Kodi on many devices, and the number of devices that Kodi supports out of the box is getting smaller.  The list for Plex, however, is getting larger. Also, the client/server Kodi implementation is a bit of a hack and only works on your local LAN.  The combination of the elegant client/server model for Plex and its vast array of native device support, lead me to test it out (among other options such as Emby), I liked it enough to switch. A perfect example of a place where Plex runs and Kodi does not (and likely will never, short of rooting the device) is on Smart TV’s. Despite being in the camp of people who do not much like Smart TV’s, as I would rather my multimedia device be decoupled from my display, for a variety of reasons, they are here to stay. Most TV’s today are smart, like it or not, and currently, Smart TV apps such as Netflix and Amazon Prime video are among the few ways one can actually consume 4k content. My new TV is a smart tv and happens to have the Plex app, which works wonderfully.  I fully anticipate not using the smart tv functions in the future as the hardware will become outdated before the display does, but my point on the portability of Plex remains the same.

So anyhow, running Plex natively on my smart tv got me to thinking: “What if my Smart TV could drive my LED setup directly?”, or at the very least, “Is there a way I can run my ambilight system from other devices like Smart TV’s or Roku’s?” As much as I am a nerd and I love my embedded Linux boxes, Roku’s and other similar devices work quite wonderfully and you don’t have to implement hacks to stream Netflix or watch Blu-rays.

So the basis of the idea is simple: take the original Philips Ambilight concept, and remove it from the TV hardware itself. Meaning, make a kit that anyone could put onto any TV, and have it be able to function for any content that TV could display. This content could be OTA broadcasts, Smart TV apps, or video coming in via HDMI, etc. Since the current and next generation of Philips HDTV’s have an ambilight system built in, the whole thing is already patented, which is probably why it is left to the hobbyist community to tinker with as opposed to seeing another manufacturer implementation. One point to hold onto for later is that the Philips TV’s use Android TV to drive their current Ambilight technology.

Current Solution Set

Today there are several ways people implement ambilight clones.  They all work, but all have the same limitations.

The first way is via ready-made kits such as the Light Berry and Lightpack. I love that people took the initiative to create these kits, however, they are simply glorified pre-packaged implementations of what I already built, with the same limitations. I will be the first in line when a kit comes out that can actually do what I want, but that is not the case with these. These kits are simply the same as my implementation but geared toward people without the time or inclination to build it themselves.

From the Lightpack FAQ
From the Lightpack FAQ

The second way is by doing something exactly like, or at least similar to, what I did. That is, use an embedded computer like a Raspberry Pi and drive the LED’s over SPI. The major differences here are in how the picture data is processed. This can be either via the network and a plugin, such as the Kodi Hyperion/Boblight plugin, or directly on the device via the Linux frame buffer in software (as I am doing today).  This is just a home brew version of the kits I just mentioned and is effectively exactly the same thing.

The third implementation is via a video capture device in-line with HDMI. This method is the absolute closest to what I want to achieve but is not something I am willing to invest time and effort into. Essentially it works like this. You get a cheap HDMI splitter capable of HDMI 1.4, 1080p video which happens to also completely ignore HDCP.  As it turns out, almost every cheap HDMI splitter on Amazon simply ignore HDCP entirely, but they don’t advertise that fact for obvious reasons.  On one side of the splitter is your TV, on the other is an HDMI-to-composite scaler that feeds a cheap USB video capture dongle connected, again, to a Raspberry Pi like device. I have a feeling there is noticeable latency involved in this setup, but that is not why I haven’t implemented it. Two years ago, I would have. The only reason I have not implemented it like this is because I want my system to be capable of the latest 4k UHD premium specs such as HDR, 10-bit color, etc. This means that I need to wait until a splitter / scaler that can ignore HDCP 2.2 as well as down sample a 4k60p signal quickly enough for low latency processing exists, or at least isn’t over $500. I am not sure how far on the horizon this might be, especially since HDFury was recently sued and told to stop selling all of their products.  While I might not agree with the idea of HDCP, I would like to not have my system be non-compliant leaving me unable to watch Netflix.

The fourth, and most elegant solution I have seen yet, is an in-line HDMI FPGA LED controller. In my mind this is what an off the shelf commercial implementation of this technology would look like, with some additional software features built in. I have only seen one implementation of this, and it is far too complicated for the average joe to implement. I would try, but it is again limited to a non HDCP compliant 1080p signal, so unless I could implement a similar device which was HDCP 2.2 / HDMI 2.0 / 4k60p compliant, it wouldn’t suit my needs. I went on a several week long research binge after I discovered this and determined that it would be completely feasible to build this with my specs in mind, I simply don’t have the skills to do it.

I even went so far as to consider trying to buy a Sil9777 chip, development board, or a Samsung SEK3500U evolution kit just to try. I just don’t think I can see myself investing that sort of time/money/effort into it, but perhaps someone else will.

The fifth, and most novel-yet-stupid implementation I have seen is to use a camera.  Some folks have used a webcam pointed at their TV screen to achieve the effect.  In fact Philips own Hue system uses this method to control the Hue bulbs with TV content.  The latency is massive and depending on what you want, this might be a great solution.  For me, it is not.  I like my ambilight to have rather high resolution and very low latency.  If you are looking for a slower, low resolution ambient light, this would be great.

My Feature Wishlist

Required Features

  • Does not require gaining unofficial root of a specific device. (Not that I have an issue with this, I just don’t want to be locked into a specific set of devices I can use.  Nor do I want to risk bricking my $3000 TV)
  • Extremely low-latency, accurate, ambilight effect from any video source.
  • HDCP 2.2 / HDMI 2.0 Compliant such that any device (such as an NVidia Shield TV or a Roku 4) can drive the LED’s without breaking HDCP.  Able to function around sources which implement UHD Premium specs such as HDR, 10-bit color, Wide gamut, etc.
  • Reliable and somewhat elegant solution, meaning, not using 8 different converters/splitters/strippers/capture devices which all rely on some software behind it all.
  • Ability to easily enable/disable the ambilight at any time.  I don’t need the ambilight effect at 2pm when I’m watching a re-run of Roseanne.
  • Implementation is relatively platform agnostic, meaning does not require a Samsung Smart TV or a Roku or any specific brand of device.
  • As portable, upgradable, and future-proof as possible.

Optional “nice to have” features

  • Software features such as app control of the lights for non-video related content.  E.g. using a smartphone app to set the lights to run in some pattern when the TV is not even on for ambience.  Hyperion can do this today.
  • Ability to respond to music instead of video, similar to a music visualizer.  I have not seen a clean implementation of this anywhere.
  • Ability to respond to content that never leaves the TV, e.g. video from a Smart TV app that does not traverse the HDMI connection. Impossible today outside of Philips TV’s.
  • Could be packaged and sold to your grandmother at best-buy, this will motivate someone to build it and make a lot of money.  This would also mean it would not be a convoluted cluster-fuck of an implementation.
  • Ability to also control additional lights, such as the Philips Hue, to control the ambient color of the entire room.  This has also been done, and Hyperion can do it.

What I Have Learned

In my research I have learned a lot, about a lot of things, which I will do my best to summarize here.  Not that anyone will read this, but if someone does, perhaps I can save you the time of digging too far into it yourself, since I already have.  Most of what I learned is that there is a ton of shit I didn’t know existed with respect to embedded systems, display technology, signal processing, etc., etc.  It has been very eye-opening.

  1. Samsung Smart TV’s, as of this year, run the Tizen OS.  LG TV’s run WebOS.  These are embedded Linux.  As such they 100% have the capability to fulfill my entire wish list, technically.  The Tizen Direct Rendering Manager spec illustrates this exact point.  Beyond that, the Tizen developer app SDK also exposes the fact that the functionality is there, but developers are artificially limited any prevented from doing it.  The Samsung Smart TV SDK “app lifecycle” prevents app developers from having apps run in the background.  It is completely possible, but unless Samsung opens the API up more, or people root Tizen on Samsung Smart TV’s, a smart tv app implementation of Hyperion will not happen.  I have even inquired with Samsung directly.
  2. Many STB’s, Smart TV’s, and entertainment boxes (such as Tivo, NVidia Shield, Roku, Amazon Fire TV, etc.) run either embedded Linux or a custom Android build.  Both of these platforms are 100% capable of running the requisite software to drive these LED setups.  Again, the block is artificial and rooting these devices would be required.  While I would be willing to buy an NVidia shield and root it, as far as I am aware that is not possible today, and it would be a very device specific implementation.  Native, non-root android apps already exist, but are proprietary.  Philips clearly developed this system for their Android based ambilight TV’s, and the Prism app that LightPack made for the OUYA are perfect examples.
  3. A hardware implementation, like the one I talked about above, would be the most ideal.  I guess the primary issue here is that if one were able to extract enough video data from the HDMI signal to do it, then they would also theoretically be able to pirate the video stream.  Given that the point of HDCP is to prevent piracy and implement DRM, I am not hopeful that the HDCP spec will ever take into consideration an idea that one might want to simply extract the edge color data from the video stream for these purposes.  As such I imagine that any hardware based implementation will have to bypass or break HDCP unless a large enough company wanted to develop this product and could actually license HDCP on the device.

Conclusion

There is a pretty massive demand for this type of setup, but not enough that the Philips Ambilight enabled TV’s are flying off the shelves.  In fact, the entire idea is not very well known by the masses but everyone who sees it in my house wants to build one.  It is my belief that whomever implements my entire wish list will be a millionaire, and very likely get sued by Philips.  I wish that person could be me, but I simply lack the abilities, time, and resources.  As a result, I hope someone out there smarter than me pulls it off, so I can buy one.  In fact, Philips should just decouple the technology from their TV’s and sell it, there is absolutely no reason they can’t do this today.  Philips TV’s are not the most popular and they could make a lot of money just embracing the idea that Samsung and LG make better TV’s, but they make pretty cool smart lights.

I am extremely frustrated by my lack of ability to come up with a reasonable solution to this problem, especially since I know it is 100% do-able but I can’t implement it due to device specific artificial restrictions or DRM based limitations.  Having a solution just outside of my reach has definitely been absolutely maddening.  All of the devices we use to consume content, right down to the cable boxes, are capable, we just don’t have access.

12 thoughts on “My Ambilight, My Obsession, My Curse

  1. Haha, oh man, this was like reading how my own brain works. I am only getting into ambilight dig following your tutorial and also worry for the future next year when I plan on getting lg OLED TV and play everything on it..
    I have also researched ways to keep the ambilight somehow but I can’t find any way.
    Actually I am surprised there is not some Plex channel which could send needed info to remote pi with the Hyperion or some other software solution.
    That would be limiting to Plex I know but it’s better than nothing. Right?

  2. I have been wondering down this rabbit hole for awhile now. I had my first iteration of ambilights going for several years now from a pi with hyperion. I’ve even gone so far as to set up a secondary pi that checks my favorite hockey teams score and upon my team scoring it will trigger a goal light effect i created. I’ve sense got my hands on a budweiser red light glass that runs a wireless imp card and works near instantaneous and sometimes faster than you see it on tv. I want to set that up to be my trigger next.

    Ambilight enhancements aside, i have recently jumped to a 4k tv. I’ve done all the necessary replacement of components and cables to be HDCP2.2 compliant. My one hang up is the downscaler to vga. My receiver recognizes its not 2.2 and limits the signal to 1080p but not 1080i and therefore my cable gets kicked to 720p. I tried a solution of placing a converter between the two that takes a 2.2 signal down to 1.4, however the handshake was still not successful and it kicked the signal down. I can’t remember where but i did read that the HD Fury Linker supposedly works, but im scared to drop 180 bucks to test.

    1. I broke down and bought an HD Fury Linker. I contacted them with my problem and they claim their product is the solution i need. It should be coming tonight and i will update on my findings.

  3. I’ve been happily enjoying a Hyperion controlled ambilight setup for several years now. I started with just RasPlex and its internal frame grabber, but eventually hated not having it for external sources so bought the splitters and went the whole hog. However, having just bought new TV for 4K and realising my receiver doesn’t have HDMI 2 so will have to rely on its inbuilt apps for 4K 60fps and HDR content, I am super bummed I won’t have my ambilight any more.

    Reading your post it is very similar to my brain chatter, even before this new problem. I love my amblight but always lusted for a more integrated solution. My brother actually works at Philips, at one point being product manager for home lighting (including Hue) and I’ve chirped at him several times to get on to the product team to decouple a Hue amblight solution that one could buy off the shelf, but so far I’m coming up empty.

    So until that happens, I’m sniffing around for other solutions and honestly my best bet right now seeing as there is no way of capturing the display of my new TV internally as it doesn’t have any signal output is to use the camera approach you dismiss. This guy actually looks to have done a really good job. http://waldobronchart.be/639/ambient-light-tv-with-a-raspberry-pi-part1/ I’m looking to recreate this but using the Pi camera, and as a source to Hyperion rather than writing my whole own solution (or run his custom software). I’ll let you know how I go.

    I’ve looked in to the API for my TV and a few others and there is nothing exposed that could grab the picture, probably so it doesn’t break the HDCP as you state.

    Eventually, with more money, I could just buy a new receiver but I’d also need a few extra dongles seeing as Prime still don’t support Chromecast, so it’s a new receiver plus a FireTV or a Roku. Not to mention new splitter that supports HDCP and HDMI 2 (https://orei.com/products/orei-1×2-2-0-hdmi-splitter-2-ports-with-full-ultra-hdcp-2-2-4k-at-60hz-3d-supports-edid-control-hdy-102).

    I do already have the RasPi kit so let’s see if I can do this just spending my time.

  4. It could be done more easily – I think – by positioning a very thin colour-sensing frame (painted acryl?) with an array of sensors over the tv frame. There should be no need to cover the lcd or oled panel itself, just have the sensors right next to it, and positioned with an inward angle. This should suffice, and allow capturing the border colors from the tv screen. Then, the software must calculate the average colour values between every two sensors, and replicate that to led colours behind the tv. The main challenge would be performance (getting a good led refresh rate) and adjusting the intensity of the leds to the intensity of the tv colours. Nowadays, local dimming would help with getting the intensity right.

  5. One possibilty is using a tv with a separate box, like a samsung connect box. Split the signal before it is sent to the tv and use that to drive the edge leds.

  6. If you are looking for a ready Ambilight system which works with everything that you can play on your TV, I recommend the AmbiVision PRO: https://ambivision.tv/
    AmbiVision uses the camera for screen tracking. I think it is the best option for people who want to have Ambilight effect, but don’t want to buy Philips 🙂

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.