How Chroma Subsampling and RGB Range Affect Colour in a Retro Gaming Signal Chain

Plug a Sega Genesis into a RetroTINK 5X-Pro and then into a 4K OLED, and the image is sharp but colour edges look wrong. Red text on a black background shimmers. A blue sprite edge develops a faint halo. The cause is not your cable or your TV. It is a compression step called chroma subsampling, sitting inside the HDMI signal chain, silently discarding colour resolution while keeping luminance sharp. For pixel art, where every red and blue edge sits against black, 4:4:4 is non-negotiable. Walk the exact signal chain you are using, understand the RGB full versus limited mismatch that crushes blacks, and change the setting on your scaler that fixes the blur before you give up and go back to composite.

Chroma subsampling is a compression scheme that exploits a weakness in human vision: you see luminance detail far more sharply than chrominance detail. A video signal stores every pixel's luminance. For colour, it shares one sample across two, four, or sixteen pixels. 4:4:4 means no sharing: every pixel gets its own colour sample. 4:2:2 shares one colour sample per two horizontally-adjacent pixels. 4:2:0 shares one colour sample per four pixels in a 2x2 block.

For a photograph of a landscape, 4:2:0 is invisible. For pixel art, it is a disaster. A single-pixel-wide red line on a black background, exactly what a 16-bit era sprite is made of, falls into a 2x2 block where three quarters of the pixels are black. The subsampled colour averages to a dark grey, and the scaler's processing spreads that grey across the whole block. The line becomes a smudge. The sharper your upscaler, the more obvious the smudge becomes, because the scaler is faithfully magnifying a mistake.

The numbers on your TV's display info menu lie. Many TVs, even high-end OLEDs, report receiving 4:4:4 while processing 4:2:0. The only way to know for sure is to examine what the scaler's output reports, or feed it a test pattern with alternating single-pixel colour bars. See grey where you should see pure red, and you have subsampling in your chain. For retro gaming, accept nothing less. Every quality scaler on the market outputs 4:4:4 at its native colour depth. The problem is not the scaler; it is what happens downstream, inside the TV's own processing or a capture card.

Your console does not output HDMI. It outputs analogue video, and the way that analogue signal becomes digital determines everything about colour accuracy. Walk the chain with a Sega Genesis, because it is the worst case for colour edge cases.

1. The Console: The Genesis's video chip outputs RGB natively. In RGB, each of the three channels is a separate analogue voltage. Red at full amplitude plus blue at full amplitude makes magenta. This is a full-range signal: black is 0V, white is 0.7V.
2. The Cable: A quality RGB SCART cable carries those three signals, plus sync, with a shield that prevents crosstalk. A cheap cable bleeds one channel into another, but that is a cable issue, not a colour space issue.
3. The Scaler: This is your analogue-to-digital converter. It samples the RGB voltage at a fixed rate, usually 27 MHz for 240p, and quantises each channel to 8 bits. The critical point is that this step is lossless in terms of colour space. RGB stays RGB.
4. The Scaler Output: Here is where it gets tricky. The RetroTINK 5X-Pro has a menu item for HDMI output. You have two choices: RGB Full Range (0-255) or RGB Limited Range (16-235). You also have the option to convert to YCbCr 4:4:4, which is mathematically lossless but introduces an extra conversion step that can round incorrectly at 8-bit.
5. The TV: The TV receives the HDMI signal and must decide how to interpret it. If the TV is set to Full and the scaler outputs Limited, blacks are crushed. If the scaler outputs Full and the TV expects Limited, blacks are lifted above black, washing out the image.

The signal chain is fragile at exactly one point: the HDMI handshake between the scaler and the TV. The EDID is a small block of metadata the TV sends to the scaler at power-on. It contains a flag that says 'I support RGB Limited, please use it.' Some TVs, particularly ones with aggressive power-saving features, lie about this flag. They say they support Full, the scaler believes them, and then the TV crushes the blacks on its own end.

Why YCbCr Exists

Analogue component video, the three-cable red/green/blue type on the back of older DVD players, is a different beast from RGB. YCbCr is the digital version of that. It encodes luminance (Y) and colour difference (Cb and Cr) separately. When your scaler converts RGB to YCbCr for HDMI output, it is not a lossless step. The conversion matrix involves multiplying the RGB values by floating-point numbers, and at 8-bit, that introduces a rounding error of 1-2 levels per channel. On a flat colour, this is invisible. On a smooth gradient, it is banding.

Your TV's Internal Bias

YCbCr is the native format of almost all video content. Blurays, streaming, broadcast TV: they are all YCbCr 4:2:0. A TV's internal processing, the part that does motion smoothing and noise reduction, is optimised for YCbCr. Feed it RGB 4:4:4, and it has to do an extra conversion internally. If the TV is cheap, that conversion is sloppy.

The Console-Side Handshake Failure

The Nintendo Switch, when set to 'Automatic' RGB range, often outputs Limited over HDMI, even when the TV supports Full. The PlayStation 5 has the same flaw. The scaler does not have this problem; the console is wrong. Your RetroTINK 5X-Pro has a menu option to force the input range, but unless you know what the console is actually sending, you are guessing. Change the console's setting to match the scaler, not the other way around.

This is the most common source of 'the image looks wrong' that is not a scaler problem. It is a metadata mismatch. RGB Full (0-255) is the native range of a PC monitor. RGB Limited (16-235) is the native range of TV broadcast. A retro console outputs 0-255 from its DAC. Run that through a scaler that outputs Limited, and you are telling the TV to map 0 to black and 235 to white. The TV obeys the signal, lifts 16 to black and leaves 235 as white. Shadows crush. Highlights blow out. But only if the TV expects the opposite.

Here is the failure mode. You have a Sony PS2 running through a RetroTINK 5X-Pro. The scaler is set to Full. The TV is a Samsung QLED, which defaults to Limited for HDMI inputs. The image looks washed out, like a grey veil has been pulled over the screen. You reach for the TV's picture controls and lower the brightness, which fixes the blacks but makes the whites grey. You are now fighting the signal chain. The fix is not a TV adjustment; it is a scaler adjustment. Change the 5X-Pro's HDMI RGB Range to Limited, and the TV displays it correctly.

How do you know which one you have? Most TVs have a 'HDMI Black Level' or 'HDMI RGB Range' option in the picture menu. On a Sony TV it is called 'HDMI Video Range'. On an LG it is 'HDMI Black Level'. Set it to 'Low' for Limited and 'High' for Full. Then set your scaler's HDMI range to match. This is the single most important control on the entire chain, and most people get it wrong because they set it once and forget, not realising a firmware update on the TV can reset it to default.

Once you have fixed the range and the subsampling, the remaining enemy is banding. Visible steps between adjacent colour values appear on smooth gradients like a sunset sky or a pre-rendered FMV background. The cause is quantisation: 8-bit video has 256 levels per channel. Take a gradient that should go smoothly from dark blue to light blue across many levels, and you have already exceeded the 8-bit limit before any processing begins. The scaler must remap those levels. If it does not dither, you get bands.

The RetroTINK 5X-Pro and the OSSC both process internally at 8-bit. Neither applies dithering. This is deliberate: dithering is a noise pattern that masks banding, but it adds noise. For pixel art, dithering is destructive because it adds a shimmer to flat colours. For a 3D game like a PS1 title, the graphics have smooth gradients, and 8-bit processing will visibly band in the skybox.

The OSSC v1.8 has a 10-bit output option over HDMI. This does not add data, but it gives the TV more headroom to do its own processing. The RetroTINK 4K goes further with 12-bit output. Here is the catch: your TV is almost certainly 8-bit. Most 4K TVs have a 10-bit panel but '8-bit plus dithering' for the processing. Feed it 12-bit, and it will dither internally to 10-bit. The practical advice: use the highest bit depth your scaler offers, but only if the TV accepts it without introducing input lag. If you see banding after that, it is the source material, not your chain.

What EDID Actually Does

The HDMI handshake is the electronic conversation that happens when you plug in a cable. The TV sends an EDID data block telling the source what formats it supports. The source, your scaler or console, reads that and decides what to output. This is how you get a resolution mismatch. EDID is not just about resolution; it carries an RGB range flag. Some TVs, particularly from Vizio and TCL, have buggy EDID implementations that always say 'Limited', even when they are set to Full. The scaler trusts the EDID, outputs Limited, and then the TV, set to Full, lifts the blacks. The result is a grey, washed-out image.

HDCP Handshake Loops

There is also the HDCP handshake. This is copy protection, not a colour control, but it can fail. A common failure is a loop where the scaler and TV keep re-negotiating, causing a blinking black screen every few seconds. A capture card in the chain is often the trigger. The capture card's HDCP engine conflicts with the TV's, and the source device gets confused. For retro gaming, HDCP is irrelevant; your console does not have it.

The practical takeaway: if you have a blinking screen or a wrong range, unplug the HDMI from the TV, wait 30 seconds, and plug it back in. This forces a new handshake. Do not just reboot the scaler; the TV's EDID ROM is what is being read.

You want the answer. Here it is. On the RetroTINK 5X-Pro, go to: Output > HDMI > RGB Range and set it to Limited (16-235). On your TV, set the 'HDMI Black Level' or equivalent to 'Low' or 'Limited'. If you are using a capture PC, set the capture software to accept Limited as well. This is the 90% solution. It fixes the washed-out blacks, it fixes the crushed shadows, and it prevents the TV from adding a gamma curve that makes pixel colours look wrong.

The remaining 10% is for the colour perfectionist who wants 4:4:4 and full range. If your TV is a PC monitor, set everything to Full. Most 4K gaming monitors are. For a TV in a living room, Limited is correct because it matches the broadcast standard. The scaler's job is not to produce the best-looking signal; it is to produce a signal the TV will not mangle. You are negotiating with a device that has its own ideas about what it is doing.

One more step: check your console's RGB range. The Nintendo Switch has a hidden control under 'System Settings > TV Settings > RGB Range'. Set it to Full while the scaler is Limited, and you have reintroduced the problem. The rule is simple: match the console to the scaler and the scaler to the TV. You are the arbiter, but you are also the bottleneck. Unwilling to spend time in the menu? Accept the default Limited, because that is what most TVs expect.

Why does my retro console look blurry on a 4K TV even with a scaler?

If the image is sharp but colours bleed at edges, chroma subsampling inside your TV's processing is the culprit, not the scaler. Force the scaler to output 4:4:4 by checking its output controls and disable any TV 'enhancement' features that might be downsampling.

What is the difference between 4:4:4 and 4:2:0 for gaming?

4:4:4 preserves all colour data per pixel; 4:2:0 discards 75% of it. For pixel art, discarded chroma turns into grey smudges on red and blue edges. For 3D games, it is usually invisible.

Should I set my TV to RGB Full or Limited?

Match the scaler. If your RetroTINK outputs Full, set the TV to Full. Unsure which the scaler outputs? Set the scaler to Limited and the TV to Low/Limited. This is the safest default.

Does the OSSC have the same subsampling problem as the RetroTINK?

No. Both the OSSC and RetroTINK 5X-Pro output 4:4:4 internally. The problem is what the TV does downstream. Neither scaler is at fault.

Why does my PlayStation 3 look washed out?

The PS3 defaults to Limited. If your TV is set to Full, you get lifted blacks. Change the PS3's 'RGB Full Range (HDMI)' to Full, or change your TV to Limited. Do not set both to Full unless you have verified the EDID says so.

What is the colour banding I see in sky gradients?

That is 8-bit quantisation, not a fixed signal chain error. The scaler processes at 8-bit. Use the OSSC's 10-bit output or the RetroTINK 4K's 12-bit if you have them, but if the TV is 8-bit, you will still see banding.

My screen blinks black every 10 seconds. What is wrong?

That is an HDCP handshake loop, not a colour issue. Unplug the HDMI for 30 seconds, or add an HDCP-compliant switch that buffers the signal. A bad EDID on the TV is the usual cause.

If you have to ask whether you can see 4:4:4, you probably cannot. The difference between 4:4:4 and 4:2:0 is visible on a test pattern but nearly invisible in gameplay unless you are playing a 2D fighting game with single-pixel red projectiles. The RGB range mismatch is a bigger problem because it affects every pixel, not just the edges. Fix the range first. If, after that, you still see a smudge on white text over black, then you have a subsampling issue. But if you are using a RetroTINK 5X-Pro or an OSSC with a quality cable, and your TV is not a budget model from 2019, you are already getting a signal that is overwhelmingly good. Do not let the spec sheet win; play the game.

The Nintendo Switch, when set to Automatic RGB range, often outputs Limited over HDMI, even when the TV supports Full, and this is a known handshake failure that requires you to force the console's RGB range setting to Full to match your scaler's output.