ZipNeRF: New AI for View Synthesis From Google Research | Explained

Google Research just released their new AI model for view synthesis, a quickly evolving research branch of computer vision. In this video, I will cover what view synthesis is and the research behind this new AI model from Google, and I will also show you how to use these models on your own. Thank you for watching! CHAPTERS ____________________ 00:00 - Intro 00:33 - What Is View Synthesis? 01:13 - Neural Radiance Field 02:48 - NeRF Extensions 03:54 - Zip-NeRF 04:50 - Instant-NGP Example PAPERS ____________________ 👉 Zip-NeRF - https://arxiv.org/pdf/2304.06706.pdf 👉 NeRF - https://dl.acm.org/doi/pdf/10.1145/3503250 👉 Instant-NGP - https://dl.acm.org/doi/pdf/10.1145/3528223.3530127 👉 Mip-NeRF - https://tinyurl.com/28ahtmry 👉 Mip-NeRF 360 - https://tinyurl.com/bddhp2fh USEFUL LINKS ____________________ 👉 Zip-NeRF website - https://jonbarron.info/zipnerf/ 👉 NeRF website - https://www.matthewtancik.com/nerf/ 👉 MipNeRF website - https://jonbarron.info/mipnerf/ 👉 MipNeRF 360 website - https://jonbarron.info/mipnerf360/ 👉 Instant-NGP Github - https://github.com/NVlabs/instant-ngp/ INSTRUCTIONS FOR INSTANT-NGP ____________________ 1️⃣ You will need an NVIDIA GPU to run Instant-NGP. If you don’t have one, you can use a virtual machine. I got a VM from Google Cloud (NVIDIA T4, 4 CPUs, 26GB mem, Debian 10) and used Chrome Remote Desktop to access the desktop. 2️⃣ Install Instant-NGP, by following the instructions on GitHub: https://github.com/nvlabs/instant-ngp 3️⃣ You should be able to use the GUI now and train the model. Test it out on their demo data: ./build/instant-ngp --scene data/nerf/fox 4️⃣ To get your own training images, follow one of the steps here: https://github.com/NVlabs/instant-ngp/blob/master/docs/nerf_dataset_tips.md#preparing-new-nerf-datasets. I tried NeRFCapture and COLMAP. COLMAP worked better but took more time to set up. 5️⃣ Then upload your images with relevant config files (e.g., transforms.json file) to your VM. And run instant-ngp on this data as in step 3. ******************** #computervision #ai #google