Jussi Rovanperä August 22, 2017 Share August 22, 2017 Hi, has anyone used DWAA/DWAB compressed EXR intead of uncompressed frames or prores/dnxhr as an intermediate codec? Could be a good solution for a frame-based workflow with UHD/4K material. However With UHD footage and timeline I get 17 fps playback in Resolve 12.5. With my current computer (4ghz quad i7) the processor usage is 80%ish, GPU is at 40%, Drive speed not a problem, 100MB/s off an SSD. I wonder what is the bottleneck with playback, probably the CPU? 1 Link to comment Share on other sites
Emily Haine August 22, 2017 Share August 22, 2017 Hi Jussi. DWAB is more efficient than DWAA because the algorithm is optimized for increased performance in most of the common post production tools. You should be able to notice the difference. 3 Link to comment Share on other sites
Jussi Rovanperä August 22, 2017 Author Share August 22, 2017 (edited) 1 hour ago, Emily Haine said: Hi Jussi. DWAB is more efficient than DWAA because the algorithm is optimized for increased performance in most of the common post production tools. You should be able to notice the difference. I get similar frame rate for both in Resolve. Piz compression gives me similar frame rate as well. Edited August 22, 2017 by Jussi Rovanperä typo Link to comment Share on other sites
Thomas Singh August 22, 2017 Share August 22, 2017 Similar frame rates suprises me. It could mean that it is a bottle neck somewhere that restricts the full potential. It could also mean that the DWAA/DWAB compression algorithm is not well enough integrated in Resolve. In Nuke the difference is huge. 3 Link to comment Share on other sites
Jussi Rovanperä August 23, 2017 Author Share August 23, 2017 10 hours ago, Thomas Singh said: Similar frame rates suprises me. It could mean that it is a bottle neck somewhere that restricts the full potential. It could also mean that the DWAA/DWAB compression algorithm is not well enough integrated in Resolve. In Nuke the difference is huge. Thanks Thomas, that's good to know. I gotta check with the Resolve 14 beta, if there is a difference to 12.5. Link to comment Share on other sites
Jussi Rovanperä August 23, 2017 Author Share August 23, 2017 (edited) Resolve 14 b7: PIZ 19.5fps DWAA 45 15fps DWAB 45 13.5fps I tried different compression options between 0-45, but the files Resolve wrote are all similar in size, so something is wrong I guess... Edited August 23, 2017 by Jussi Rovanperä typo 1 Link to comment Share on other sites
Nicolas Hanson August 23, 2017 Share August 23, 2017 EXR also runs slow on my system and I tend to ask for DPX if VFX want to deliver image sequences. I don't know how Resolve extracts EXR data, but in Lustre you need to apply a Float LUT to access out of range information. 1 Link to comment Share on other sites
Nicolas Hanson August 23, 2017 Share August 23, 2017 1 hour ago, Jussi Rovanperä said: Resolve 14 b7: PIZ 19.5fps DWAA 45 15fps DWAB 45 13.5fps I tried different compression options between 0-45, but the files Resolve wrote are all similar in size, so something is wrong I guess... Maybe @Peter Chamberlain can have a look at this? 1 Link to comment Share on other sites
Jussi Rovanperä August 23, 2017 Author Share August 23, 2017 (edited) Scratch Play can playback the PIZ / DWAA and DWAB files @25fps EDIT: Sorry, that was a bit too optimistic, Scratch Play seems to cache single clip playback. With Scratch I get realtime (25fps) for PIZ (cpu at 80%) , 22fps for DWAA and 20fps for DWAB (cpu at 100%). With Resolve the cpu doesn't go to 100%, but stays at 80%. Edited August 23, 2017 by Jussi Rovanperä misinformation 1 Link to comment Share on other sites
Emily Haine August 23, 2017 Share August 23, 2017 (edited) 5 hours ago, Jussi Rovanperä said: With Scratch I get realtime (25fps) for PIZ (cpu at 80%) , 22fps for DWAA and 20fps for DWAB (cpu at 100%). The way I thought Scratch worked was reading DWAB data by bundling all the lines which should mean increased speed. Can you comment on this @Mazze? Edited August 23, 2017 by Emily Haine 1 Link to comment Share on other sites
Tom Evans August 30, 2017 Share August 30, 2017 Any progress on this? Link to comment Share on other sites
Mazze August 31, 2017 Share August 31, 2017 On 23.8.2017 at 6:29 PM, Emily Haine said: The way I thought Scratch worked was reading DWAB data by bundling all the lines which should mean increased speed. Can you comment on this @Mazze? Not 100% like this, but we read the full file into memory as fast as possible and from there start the decode as fast as possible. However, we don't need all the file data to be loaded already in order to start the decode :-) . 1 Link to comment Share on other sites
Abby Bader September 1, 2017 Share September 1, 2017 Mazze, can you say something about the time difference in decoding DWAA vs DWAB footage? Link to comment Share on other sites
Emily Haine September 17, 2017 Share September 17, 2017 On 8/31/2017 at 4:29 PM, Mazze said: Not 100% like this, but we read the full file into memory as fast as possible and from there start the decode as fast as possible. However, we don't need all the file data to be loaded already in order to start the decode :-) . Does that mean DWAB is processed faster? Link to comment Share on other sites
Johannes Møgelvang September 19, 2017 Share September 19, 2017 (edited) Hi Guys. I made some tests about this some time ago for our workflow. I tested render speeds of the same shot from Scratch to different formats and compression's: Write speeds: 12s - DPX (16.56 fps) 18s - EXR - B44 (11.68 fps) 18s - EXR - B44A (11.72 fps) 19s - EXR - DWAA (10.66 fps) 20s - EXR - RLE (10.51 fps) 21s - EXR - none (10.05 fps) 22s - EXR - ZIP (9.61 fps) 26s - EXR - ZIPS (8.06 fps) 26s - EXR - PIZ (8.13 fps) 30s - EXR - DWAB (6.94 fps) 33s - EXR - PXR24 (6.28 fps) -And read speeds in nuke of the frames from scratch: (the results here is only for one frame, but I did the test for 100 frames as well. For 100 frames the numbers are just higher all around, but the resulting order is the same, and the smaller numbers are easier to relate to) Read speeds: 53ms - EXR none 54ms - DPX 65ms - EXR RLE 87ms - EXR ZIP 302ms - EXR DWAA 380ms - EXR PXR24 398ms - EXR ZIPS 404ms - EXR B44 411ms - EXR B44a 471ms - EXR PIZ 847ms - EXR DWAB -and file sizes also played a part in our evaluation: We decided to use EXR DWAA as our all round got to format as it seems to suit our needs and is quite good all around. If we need super high quality we go with PIZ or ZIP. I know thees read and write speeds is not the same for all tools and it might also depend on hardware, but it is what I needed for our pipeline. Cheers Johs Edited September 19, 2017 by Johannes Møgelvang 4 Link to comment Share on other sites
Emily Haine September 20, 2017 Share September 20, 2017 Very interesting and comprehensive 'report', and I'm very surprised by the results. Link to comment Share on other sites
Mazze September 21, 2017 Share September 21, 2017 Just a little update on this - the latest SCRATCH build has a fix for EXR rendering, which also has an impact on reading EXRs that have been rendered with SCRATCH: Until now, EXR renders did not always include the line offset table correctly, so when reading those renders back in, the table needed to be created manually, which could cause a playback performance drop for high resolution files. Possibly, if you now re-run the tests with EXRs generated from the latest SCRATCH build, you might see an improvement on the playback side. Cheers, Mazze Link to comment Share on other sites
