Last year, we covered one man’s efforts to upscale Star Trek: Deep Space Nine using Topaz Gigapixel AI. If you’ve ever attempted to watch DS9 on Netflix or Amazon, you know what a shame that is. The version of DS9 that you can stream via online services sucks, to put it kindly. The streamed version of the show ranges from slightly worse to much worse than what you’d get from an MKV rip or original DVD. Even if you own the original DVDs, however, DS9’s quality leaves much to be desired.
There are a number of shows that were shot from the early 1990s – early 2000s that didn’t broadcast in HD and were never upconverted to 720p or 1080p. If you love series like Babylon 5, Voyager, or Deep Space Nine, you’ve had few-to-no options for a decent viewing experience… until now. Topaz Labs’ new Video Enhance AI software ($299) uses artificial intelligence to upscale low-resolution video in ways that can dramatically improve on the original image. I’ve spent the last few weeks taking the application for a test drive.
While Gigapixel AI can upscale content, there are actually multiple reasons not to use it for the task. The app will not allow you to load large numbers of images at once and in the past, it’s had a bad habit of crashing every 500 – 600 frames. Considering there are roughly 66,000 frames in an episode, it looked as if improving DS9’s image quality would require an absolutely insane amount of work.
Way of the Warrior upscaling in progress. Click all images to enlarge.
Then, a few weeks ago, Topaz released a new product, Video Enhance AI. Instead of literally unpacking a TV show into each individual frame and then upscaling them one by one, Video Enhance AI is designed to convert entire video files at once. Performance on a GTX 2080 is roughly 95 frames per minute, or approximately one episode every 10.5 hours. While this represents a non-trivial time commitment to rip several seasons, it’s vastly better than Gigapixel AI’s demonstrated performance. Whether GP produces better results or if these results can be further enhanced via application of Topaz AI’s other software suites is something I intend to explore in greater detail in future articles. For now, we’re focusing on Topaz Video Enhance AI, specifically.
How Topaz Video Enhance AI Works
Topaz VE allows you to choose the quality level of your source from three options: High Quality, Low Quality, and HQ-CGI. HQ-CGI boosts anti-aliasing in CGI scenes, and I also ran some specific tests of this setting on the test episode I converted and will be demonstrating today. On a show like Deep Space Nine, you’ll definitely want to use LQ — a 720×480 initial input is basically the poster-child for a low-quality upscale. If you were trying to scale 1080p video up to 4K or 8K, you’d want to use the HQ setting. The application has very few options beyond choosing your file output name, your desired level of upscaling (200 percent and 400 percent were both tested here), and what file format you want to output in. Choose your starting and ending frames, hit the button, and away you go.
Be advised, however, that Topaz Video Enhance AI isn’t capable of passing audio to the newly encoded file in at least some cases. All of our test encodes came out sans audio. We recombined the audio streams into the primary video file using FFMPEG.
How We Tested
I’ve performed all of my testing with MKV files I created from the DS9 boxed set years ago. After seeing how this project came out, I actually intend to dust off my old DVDs (when I can find them) and try this project directly on the DVD source itself. ExtremeTech does not endorse piracy or recommend stealing Deep Space Nine or any other television show. If you’re interested in this kind of upscaling, you should buy the DVDs. It’s entirely possible you should do that anyway, given that the source will almost certainly be a better target for this kind of scaling.
I’ve performed a number of test runs and gathered comparison data from HQ-CGI versus LQ, as well as a wide range of Netflix – MKV – Upscaled comparisons at both the 2x and 4x scaling factors. I’ve also included a range of screenshots to show the benefit of increasing the default brightness a bit even if the DVD source is all you have. And we’ve got some YouTube uploads for you, to show the benefits of the upscaling using the DS9 credits.
Let’s get started. For starters, here’s the Defiant under attack in three frames from “Way of the Warrior.” These three are not frame-matched identically, so focus on the architecture and clarity of the ship, not the disruptor fire striking it. This is Netflix, MKV, and 4x upscale, in that order. All images can be clicked to enlarge in a new window.
Netflix, in all its smeary glory. The phrase “USS Defiant” around the saucer edge looks more like a row of windows.
The MKV rip based on DVD source. The shot is much clearer and “USS Defiant” is legible, but the overall quality still isn’t very good.
Finally, our 4x upscale. The quality level here still isn’t truly HD, but it’s vastly better than anything in the previous frame.
These were some of the first tests I did, and I was quite impressed with the results. Let’s look at a larger cross-section of an episode. Click on each image to enlarge (and you’ll want to, in order to see the degree of improvement).
Call to Arms Comparisons
All of the following screenshots and comparisons are taken from “A Call to Arms”, the 26th episode of the fifth season. I’ve included both 200 percent and 400 percent comparison images in this collection to illustrate the trade-off from each size. The best balance between visual quality and image size that I’ve seen is to upscale either 200 percent from the original DVD or to upscale 400 percent and then watch the stream at half-size.
Obviously most of you are going to be reading this on monitors, not TVs, but I checked the output quality on a standard TV set as well. If you’re sitting at standard viewing distances, all of the versions look better, but the 400 percent upscale benefits the most. I’ll call out which comparisons are 2x scaling and which are 4x scaling comparisons. All Netflix comparisons are 4x, because Netflix doesn’t allow for resolution-based resizing.
I’m going to cover image comparisons first, before I hit the video comparisons.
MKV versus 2x/4x Upscaling
Weyoun docked at Deep Space Nine. MKV.
Weyoun docked at Deep Space Nine. 4x Upscale.
The amount of detail recovered from this scene compared with the MKV is impressive. There’s a smeary sort of vaseline look to the MKV file that isn’t present in the upscaled version.
At just 2x zoom, the original MKV doesn’t look so bad…
But the improvement in the upscale is still very impressive.
Video Enhance does strong work with the starship battles in Deep Space Nine. The front of the Jem’Hadar battleship looks downright sharp. Compares 4x upscaling displayed at 1:2 ratio to standard MKV displayed at 2x size.
Netflix versus MKV versus 4x Upscaling
This screenshot of Deep Space Nine under heavy fire is a poster child for why the Netflix version of the TV show deserves to be drug out into the street and shot. The entire frame is noticeably orange-r (and these frames are matched — you can tell by the exact position of the pieces of debris in the lower-right-corner, inside the fireball).
The MKV original dumps the reddish lighting and looks a bit less blurred, even though it has the same base resolution. Color balance is better.
Our 4x upscale. The station and explosions both benefit enormously.
Next up: A comparison of the same Jem’Hadar battlecruiser that serves as a poster child for why the Netflix version of the TV show… wait. I already said that, didn’t I?
Well, it’s still deserved. Here’s Netflix.
Check the aliasing on the left wing of the battlecruiser. It looks like it was rendered in Quake II, which is impressive, considering DS9 is still using models at this point in time.
Here’s the DVD. Much better as far as the left-hand wing, and better overall preservation of detail, but still not great compared with what we’d want in a modern show.
Much, much better. Quite good, in fact. Alright. We’ve done two battle scenes — let’s take a look at how some of the character shots come out.
Weyoun’s unctuous used-car salesman vibe made him a great character foil for Gul Dukat. Netflix is still the blurriest option of the lot, but the gap between it and the MKV file is much smaller than normal.
Here’s the MKV file. Not much difference, though it’s minimally clearer and doesn’t have the weird color issue Netflix does.
Again, the upscale does a good job of drawing out subtle detail in Weyoun’s eyes and face. His jacket texturing is also much less blurry.
Improve Image Quality by Bumping Brightness
The color reproduction in the Netflix version of DS9 is terrible and the MKV just isn’t much better, but a small tap to brightness can improve the situation, in my opinion — and this option doesn’t require any upscaling or processing time. Here’s the Netflix version, followed by an MKV screenshot.
Netflix. Slightly brighter, but less detail.
MKV file. Dim and rather poor-looking, but slightly more detail.
Nudge brightness up a bit in-player, and you get this:
Rome — Actually Visible Edition. (season to taste)
Whether or not you consider this an improvement is in the eye of the beholder, but many DS9 episodes are much darker than I remember them being when I watched them the first time, and this tweak doesn’t require anything but a software player like VLC that supports it. Finally, here’s the same image upscaled 400 percent in Topaz VE.
Check his jacket texture compared with the image above.
I’ve also tossed a number of images into the slideshow below, with both space battles and additional characters. Check them out if you’re curious. The slideshow does a nice job of showing subtle improvements when it shifts from one image to the next.
Opening Credits
If you’d like to see what the opening credits to the show look like when rendered in 200 percent and 400 percent upscaling compared with the standard version, I’ve embedded YouTube links to all three below. Note that while the first link isn’t mine and does look slightly worse than what you’d see on Netflix, it’s honestly pretty close. Make sure to set the second and third links to 720p and 1080p, respectively. Both of these encodes were done using the HQ-CGI mode.
The standard introduction. Full-screen for best comparison — and this is only very slightly worse than what is available on Netflix.
200 percent AI upscaling using the HQ-CGI preset instead of LQ. HQ-CGI yields better results on some special effects, though I’m still working out what the subtleties are. The 200 percent upscale is subtly different than the 400 percent. Set to 720p for best comparison.
The 400 percent upscale. Set to 2160p for best comparison.
Discussion and Analysis
If I’m being honest, the existing encode quality I’m seeing is about 85 percent of where I’d like it to be. I’ve already found myself wondering if some of Topaz AI’s other tools might be deployed to perform some additional post-processing where it’s needed. The application struggles with graphics displayed on terminals or PADs, and I’m still testing to see if that can be resolved within Topaz Video Enhance AI or not.
The MKV version. Obviously not everything from the mid-1990s has aged well.
This is one place where the upscaling effect *doesn’t* improve image quality.
I’m not going to claim that upscaling like this simply makes Deep Space Nine look like it was shot in modern times. The color reproduction is bad in all cases and no matter how good an upscaling algorithm is, it’s still an upscaler — and therefore not the same as having the original data via 35mm film. These are all fair complaints. There are still some places where I’m hoping to clear up these files further. But the output I’ve gotten is leaps and bounds ahead of the MKV versions I’ve got, and vastly better than the Netflix streams. I’m already planning to encode most of the show — I just want to see how much additional improvement I can squeeze out first.
On a big-screen TV, Deep Space Nine is barely watchable via Netflix or Amazon. The old MKV rips I did years ago aren’t much better, which is one reason I’m going to return to original source for my next round of tests. But while I may not have a perfect solution to present day, Topaz Video Enhance AI has taken the idea from “Maybe someday,” to “Holy crap, this works now.”
Low quality source for DS9 is typically better than HQ source as far as I’ve seen, but I’m still checking if HQ-CGI can be spliced into an MP4 that uses LQ for everything else. I may also check Gigapixel again, just to see how the AI upscaling method compares, or if other Topaz Labs products can improve the final output further.
Performance-wise, both AMD and Intel systems perform identically when you test using a GPU — the RTX 2080
averaged between 95 – 105fpm (frames per minute). CPU encoding is 10x slower than GPU encoding, so I’ve only begun testing that mode. GPU encoding has a reputation for not being quite as good as CPU encode in applications like Handbrake, but I haven’t yet seen evidence it’s true in Topaz Video Enhance AI. Even if CPUs do give better results, 9-10 frames per minute would mean 110 hours per episode. Even if the quality jump was dramatic, I’d only use the CPU for trouble spots — the encode time is just too long otherwise.
These software packages and techniques have significant applicability to shows beyond Star Trek. Series like Buffy the Vampire Slayer were converted for HD, but fans hated the way color balance and the 16:9 conversion were both handled. Babylon 5 can’t be remastered due to massive rights entanglements and the destruction of its original assets, which means an upscale project like this would be its best bet.
I intend to write more on this topic in the future, but I hope you’ve enjoyed the preview. Topaz Video Enhance AI and other Topaz products are available for 30-day free trials, which is how I tested the application. You can also purchase it at the current introductory price of $199, down from $299 standard. If you’re a serious cinephile who enjoys this kind of editing, it’s worth it.
Now Read:
- Fan Works to Remaster Star Trek: Deep Space Nine in 1080p HD, Using AI
First Trailer Arrives for Upcoming Star Trek: Picard on CBS
Astronomers Find Exoplanet In the Same Place as Star Trek’s Vulcan
from ExtremeTechExtremeTech https://www.extremetech.com/extreme/306701-upscaling-star-trek-deep-space-nine-using-topaz-video-enhance-ai-review
No comments:
Post a Comment