‘CLIP’ directory

Annotations sorted by machine learning into inferred 'tags'. This provides an alternative way to browse: instead of by date order, one can browse in topic order. The 'sorted' list has been automatically clustered into multiple sections & auto-labeled for easier browsing.

Beginning with the newest annotation, it uses the embedding of each annotation to attempt to create a list of nearest-neighbor annotations, creating a progression of topics. For more details, see the link.

Miscellaneous

Bibliography

https://research.google/blog/taking-medical-imaging-embeddings-3d/: “CT Foundation: Taking Medical Imaging Embeddings 3D ”, Atilla Kiraly, Madeleine Traverse

link-bibliography
https://arxiv.org/abs/2408.05446: “Ensemble Everything Everywhere: Multi-Scale Aggregation for Adversarial Robustness ”, Stanislav Fort, Balaji Lakshminarayanan

link-bibliography
https://arxiv.org/abs/2405.07425: “Sakuga-42M Dataset: Scaling Up Cartoon Research ”, Zhenglin Pan, Yu Zhu, Yuxuan Mu

link-bibliography
https://arxiv.org/abs/2405.02793#google: “ImageInWords: Unlocking Hyper-Detailed Image Descriptions ”, Roopal Garg, Andrea Burns, Burcu Karagol Ayan, Yonatan Bitton, Ceslee Montgomery, Yasumasa Onoe, Andrew Bunner, Ranjay Krishna, Jason Baldridge, Radu Soricut

link-bibliography
https://arxiv.org/abs/2404.01291: “Evaluating Text-To-Visual Generation With Image-To-Text Generation ”, Zhiqiu Lin, Deepak Pathak, Baiqi Li, Jiayao Li, Xide Xia, Graham Neubig, Pengchuan Zhang, Deva Ramanan

link-bibliography
2024-vong.pdf: “Grounded Language Acquisition through the Eyes and Ears of a Single Child ”, Wai Keen Vong, Wentao Wang, A. Emin Orhan, Brenden M. Lake

link-bibliography
https://arxiv.org/abs/2312.16862: “TinyGPT-V: Efficient Multimodal Large Language Model via Small Backbones ”, Zhengqing Yuan, Zhaoxu Li, Lichao Sun

link-bibliography
https://arxiv.org/abs/2312.11556: “StarVector: Generating Scalable Vector Graphics Code from Images ”, Juan A. Rodriguez, Shubham Agarwal, Issam H. Laradji, Pau Rodriguez, David Vazquez, Christopher Pal, Marco Pedersoli

link-bibliography
https://arxiv.org/abs/2312.05328#deepmind: “Bad Students Make Great Teachers: Active Learning Accelerates Large-Scale Visual Understanding ”, Talfan Evans, Shreya Pathak, Hamza Merzic, Jonathan Schwarz, Ryutaro Tanno, Olivier J. Henaff

link-bibliography
https://arxiv.org/abs/2309.17425#apple: “Data Filtering Networks ”, Alex Fang, Albin Madappally Jose, Amit Jain, Ludwig Schmidt, Alexander Toshev, Vaishaal Shankar

link-bibliography
https://arxiv.org/abs/2309.16671: “Demystifying CLIP Data ”, Hu Xu, Saining Xie, Xiaoqing Ellen Tan, Po-Yao Huang, Russell Howes, Vasu Sharma, Shang-Wen Li, Gargi Ghosh, Luke Zettlemoyer, Christoph Feichtenhofer

link-bibliography
https://arxiv.org/abs/2307.01952#stability: “SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis ”, Dustin Podell, Zion English, Kyle Lacey, Andreas Blattmann, Tim Dockhorn, Jonas Müller, Joe Penna, Robin Rombach

link-bibliography
https://arxiv.org/abs/2306.15658: “CLIPA-V2: Scaling CLIP Training With 81.1% Zero-Shot ImageNet Accuracy within a $10,000 Budget; An Extra $4,000 Unlocks 81.8% Accuracy ”, Xianhang Li, Zeyu Wang, Cihang Xie

link-bibliography
2023-yi.pdf: “Anime Character Identification and Tag Prediction by Multimodality Modeling: Dataset and Model ”, Fan Yi, Jiaxiang Wu, Minyi Zhao, Shuigeng Zhou

link-bibliography
https://arxiv.org/abs/2306.09346: “Rosetta Neurons: Mining the Common Units in a Model Zoo ”, Amil Dravid, Yossi Gandelsman, Alexei A. Efros, Assaf Shocher

link-bibliography
2023-samo.pdf: “Artificial Intelligence and Art: Identifying the Esthetic Judgment Factors That Distinguish Human & Machine-Generated Artwork ”, Andrew Samo, Scott Highhouse

link-bibliography
https://arxiv.org/abs/2305.16934: “On Evaluating Adversarial Robustness of Large Vision-Language Models ”, Yunqing Zhao, Tianyu Pang, Chao Du, Xiao Yang, Chongxuan Li, Ngai-Man Cheung, Min Lin

link-bibliography
https://arxiv.org/abs/2305.07017: “An Inverse Scaling Law for CLIP Training ”, Xianhang Li, Zeyu Wang, Cihang Xie

link-bibliography
https://arxiv.org/abs/2305.05665#facebook: “ImageBind: One Embedding Space To Bind Them All ”, Rohit Girdhar, Alaaeldin El-Nouby, Zhuang Liu, Mannat Singh, Kalyan Vasudev Alwala, Armand Joulin, Ishan Misra

link-bibliography
https://arxiv.org/abs/2305.01569: “Pick-A-Pic: An Open Dataset of User Preferences for Text-To-Image Generation ”, Yuval Kirstain, Adam Polyak, Uriel Singer, Shahbuland Matiana, Joe Penna, Omer Levy

link-bibliography
https://arxiv.org/abs/2304.07193#facebook: “DINOv2: Learning Robust Visual Features without Supervision ”, Maxime Oquab, Timothée Darcet, Théo Moutakanni, Huy Vo, Marc Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaaeldin El-Nouby, Mahmoud Assran, Nicolas Ballas, Wojciech Galuba, Russell Howes, Po-Yao Huang, Shang-Wen Li, Ishan Misra, Michael Rabbat, Vasu Sharma, Gabriel Synnaeve, Hu Xu, Hervé Jegou, Julien Mairal, Patrick Labatut, Armand Joulin, Piotr Bojanowski

link-bibliography
https://arxiv.org/abs/2304.05538: “ImageNet-Hard: The Hardest Images Remaining from a Study of the Power of Zoom and Spatial Biases in Image Classification ”, Mohammad Reza Taesiri, Giang Nguyen, Sarra Habchi, Cor-Paul Bezemer, Anh Nguyen

link-bibliography
https://arxiv.org/abs/2303.15343#google: “Sigmoid Loss for Language Image Pre-Training ”, Xiaohua Zhai, Basil Mustafa, Alexander Kolesnikov, Lucas Beyer

link-bibliography
https://openreview.net/forum?id=KRLUvxh8uaX: “When and Why Vision-Language Models Behave like Bags-Of-Words, and What to Do About It? ”, Mert Yuksekgonul, Federico Bianchi, Pratyusha Kalluri, Dan Jurafsky, James Zou

link-bibliography
https://arxiv.org/abs/2301.12597#salesforce: “BLIP-2: Bootstrapping Language-Image Pre-Training With Frozen Image Encoders and Large Language Models ”, Junnan Li, Dongxu Li, Silvio Savarese, Steven Hoi

link-bibliography
https://arxiv.org/abs/2301.07088#bytedance: “MUG: Vision Learners Meet Web Image-Text Pairs ”, Bingchen Zhao, Quan Cui, Hao Wu, Osamu Yoshie, Cheng Yang

link-bibliography
https://laion.ai/blog/giant-openclip/: “Reaching 80% Zero-Shot Accuracy With OpenCLIP: VIT-G/14 Trained On LAION-2B ”, Mitchell Wortsman

link-bibliography
https://arxiv.org/abs/2212.07143: “Reproducible Scaling Laws for Contrastive Language-Image Learning ”, Mehdi Cherti, Romain Beaumont, Ross Wightman, Mitchell Wortsman, Gabriel Ilharco, Cade Gordon, Christoph Schuhmann, Ludwig Schmidt, Jenia Jitsev

link-bibliography
https://arxiv.org/abs/2212.06138#microsoft: “CLIP Itself Is a Strong Fine-Tuner: Achieving 85.7% and 88.0% Top-1 Accuracy With ViT-B and ViT-L on ImageNet ”, Xiaoyi Dong, Jianmin Bao, Ting Zhang, Dongdong Chen, Shuyang Gu, Weiming Zhang, Lu Yuan, Dong Chen, Fang Wen, Nenghai Yu

link-bibliography
https://arxiv.org/abs/2211.12561#facebook: “Retrieval-Augmented Multimodal Language Modeling ”, Michihiro Yasunaga, Armen Aghajanyan, Weijia Shi, Rich James, Jure Leskovec, Percy Liang, Mike Lewis, Luke Zettlemoyer, Wen-tau Yih

link-bibliography
https://openreview.net/forum?id=wmGlMhaBe0: “MaskDistill: A Unified View of Masked Image Modeling ”, Anonymous

link-bibliography
https://arxiv.org/abs/2211.07292: “Paella: Fast Text-Conditional Discrete Denoising on Vector-Quantized Latent Spaces ”, Dominic Rampas, Pablo Pernias, Elea Zhong, Marc Aubreville

link-bibliography
https://arxiv.org/abs/2211.06679#baai: “AltCLIP: Altering the Language Encoder in CLIP for Extended Language Capabilities ”, Zhongzhi Chen, Guang Liu, Bo-Wen Zhang, Fulong Ye, Qinghong Yang, Ledell Wu

link-bibliography
https://arxiv.org/abs/2211.01324#nvidia: “EDiff-I: Text-To-Image Diffusion Models With an Ensemble of Expert Denoisers ”, Yogesh Balaji, Seungjun Nah, Xun Huang, Arash Vahdat, Jiaming Song, Karsten Kreis, Miika Aittala, Timo Aila, Samuli Laine, Bryan Catanzaro, Tero Karras, Ming-Yu Liu

link-bibliography
https://www.biorxiv.org/content/10.1101/2022.09.27.508760.full: “Incorporating Natural Language into Vision Models Improves Prediction and Understanding of Higher Visual Cortex ”, Aria Y. Wang, Kendrick Kay, Thomas Naselaris, Michael J. Tarr, Leila Wehbe

link-bibliography
https://arxiv.org/abs/2209.03953: “Fast Text2StyleGAN: Text-Free Learning of a Natural Language Interface for Pretrained Face Generators ”, Xiaodan Du, Raymond A. Yeh, Nicholas Kolkin, Eli Shechtman, Greg Shakhnarovich

link-bibliography
https://arxiv.org/abs/2209.03320: “What Does a Platypus Look Like? Generating Customized Prompts for Zero-Shot Image Classification (CuPL) ”, Sarah Pratt, Rosanne Liu, Ali Farhadi

link-bibliography
https://arxiv.org/abs/2208.12266#facebook: “Decoding Speech from Non-Invasive Brain Recordings ”, Alexandre Défossez, Charlotte Caucheteux, Jérémy Rapin, Ori Kabeli, Jean-Rémi King

link-bibliography
https://arxiv.org/abs/2208.05516: “Quality Not Quantity: On the Interaction between Dataset Design and Robustness of CLIP ”, Thao Nguyen, Gabriel Ilharco, Mitchell Wortsman, Sewoong Oh, Ludwig Schmidt

link-bibliography
https://arxiv.org/abs/2208.03550: “EVL: Frozen CLIP Models Are Efficient Video Learners ”, Ziyi Lin, Shijie Geng, Renrui Zhang, Peng Gao, Gerard de Melo, Xiaogang Wang, Jifeng Dai, Yu Qiao, Hongsheng Li

link-bibliography
https://arxiv.org/abs/2207.14525: “TOnICS: Curriculum Learning for Data-Efficient Vision-Language Alignment ”, Tejas Srinivasan, Xiang Ren, Jesse Thomason

link-bibliography
https://arxiv.org/abs/2207.12661#microsoft: “MS-CLIP: Learning Visual Representation from Modality-Shared Contrastive Language-Image Pre-Training ”, Haoxuan You, Luowei Zhou, Bin Xiao, Noel Codella, Yu Cheng, Ruochen Xu, Shih-Fu Chang, Lu Yuan

link-bibliography
https://arxiv.org/abs/2207.13061: “NewsStories: Illustrating Articles With Visual Summaries ”, Reuben Tan, Bryan A. Plummer, Kate Saenko, J. P. Lewis, Avneesh Sud, Thomas Leung

link-bibliography
https://arxiv.org/abs/2207.07285#alibaba: “X-CLIP: End-To-End Multi-Grained Contrastive Learning for Video-Text Retrieval ”, Yiwei Ma, Guohai Xu, Xiaoshuai Sun, Ming Yan, Ji Zhang, Rongrong Ji

link-bibliography
https://arxiv.org/abs/2207.07635: “Is a Caption Worth a Thousand Images? A Controlled Study for Representation Learning ”, Shibani Santurkar, Yann Dubois, Rohan Taori, Percy Liang, Tatsunori Hashimoto

link-bibliography
https://arxiv.org/abs/2207.04429: “LM-Nav: Robotic Navigation With Large Pre-Trained Models of Language, Vision, and Action ”, Dhruv Shah, Blazej Osinski, Brian Ichter, Sergey Levine

link-bibliography
https://arxiv.org/abs/2205.16007#microsoft: “Improved Vector Quantized Diffusion Models ”, Zhicong Tang, Shuyang Gu, Jianmin Bao, Dong Chen, Fang Wen

link-bibliography
https://arxiv.org/abs/2205.14459: “CyCLIP: Cyclic Contrastive Language-Image Pretraining ”, Shashank Goel, Hritik Bansal, Sumit Bhatia, Ryan A. Rossi, Vishwa Vinay, Aditya Grover

link-bibliography
https://arxiv.org/abs/2205.10747: “VidIL: Language Models With Image Descriptors Are Strong Few-Shot Video-Language Learners ”, Zhenhailong Wang, Manling Li, Ruochen Xu, Luowei Zhou, Jie Lei, Xudong Lin, Shuohang Wang, Ziyi Yang, Chenguang Zhu, Derek Hoiem, Shih-Fu Chang, Mohit Bansal, Heng Ji

link-bibliography
https://arxiv.org/abs/2205.08535: “AvatarCLIP: Zero-Shot Text-Driven Generation and Animation of 3D Avatars ”, Fangzhou Hong, Mingyuan Zhang, Liang Pan, Zhongang Cai, Lei Yang, Ziwei Liu

link-bibliography
https://arxiv.org/abs/2205.01917#google: “CoCa: Contrastive Captioners Are Image-Text Foundation Models ”, Jiahui Yu, Zirui Wang, Vijay Vasudevan, Legg Yeung, Mojtaba Seyedhosseini, Yonghui Wu

link-bibliography
https://arxiv.org/abs/2205.01397: “Data Determines Distributional Robustness in Contrastive Language Image Pre-Training (CLIP) ”, Alex Fang, Gabriel Ilharco, Mitchell Wortsman, Yuhao Wan, Vaishaal Shankar, Achal Dave, Ludwig Schmidt

link-bibliography
https://arxiv.org/pdf/2204.06125#page=16&org=openai: “DALL·E 2: Hierarchical Text-Conditional Image Generation With CLIP Latents § 7. Limitations and Risks ”, Aditya A. Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, Mark Chen

link-bibliography
https://arxiv.org/abs/2204.05080#deepmind: “Semantic Exploration from Language Abstractions and Pretrained Representations ”, Allison C. Tam, Neil C. Rabinowitz, Andrew K. Lampinen, Nicholas A. Roy, Stephanie C. Y. Chan, D. J. Strouse, Jane X. Wang, Andrea Banino, Felix Hill

link-bibliography
https://arxiv.org/abs/2204.03610#microsoft: “Unified Contrastive Learning in Image-Text-Label Space ”, Jianwei Yang, Chunyuan Li, Pengchuan Zhang, Bin Xiao, Ce Liu, Lu Yuan, Jianfeng Gao

link-bibliography
https://arxiv.org/abs/2204.00598#google: “Socratic Models: Composing Zero-Shot Multimodal Reasoning With Language ”, Andy Zeng, Adrian Wong, Stefan Welker, Krzysztof Choromanski, Federico Tombari, Aveek Purohit, Michael Ryoo, Vikas Sindhwani, Johnny Lee, Vincent Vanhoucke, Pete Florence

link-bibliography
https://arxiv.org/abs/2203.11096: “CLIP Meets GamePhysics: Towards Bug Identification in Gameplay Videos Using Zero-Shot Transfer Learning ”, Mohammad Reza Taesiri, Finlay Macklon, Cor-Paul Bezemer

link-bibliography
https://arxiv.org/abs/2203.05482: “Model Soups: Averaging Weights of Multiple Fine-Tuned Models Improves Accuracy without Increasing Inference Time ”, Mitchell Wortsman, Gabriel Ilharco, Samir Yitzhak Gadre, Rebecca Roelofs, Raphael Gontijo-Lopes, Ari S. Morcos, Hongseok Namkoong, Ali Farhadi, Yair Carmon, Simon Kornblith, Ludwig Schmidt

link-bibliography
https://arxiv.org/abs/2202.06767#huawei: “Wukong: 100 Million Large-Scale Chinese Cross-Modal Pre-Training Dataset and A Foundation Framework ”, Jiaxi Gu, Xiaojun Meng, Guansong Lu, Lu Hou, Minzhe Niu, Hang Xu, Xiaodan Liang, Wei Zhang, Xin Jiang, Chunjing Xu

link-bibliography
https://arxiv.org/abs/2201.12086#salesforce: “BLIP: Bootstrapping Language-Image Pre-Training for Unified Vision-Language Understanding and Generation ”, Junnan Li, Dongxu Li, Caiming Xiong, Steven Hoi

link-bibliography
https://arxiv.org/abs/2201.08371#facebook: “SWAG: Revisiting Weakly Supervised Pre-Training of Visual Perception Models ”, Mannat Singh, Laura Gustafson, Aaron Adcock, Vinicius de Freitas Reis, Bugra Gedik, Raj Prateek Kosaraju, Dhruv Mahajan, Ross Girshick, Piotr Dollár, Laurens van der Maaten

link-bibliography
https://arxiv.org/abs/2201.07520#facebook: “CM3: A Causal Masked Multimodal Model of the Internet ”, Armen Aghajanyan, Bernie Huang, Candace Ross, Vladimir Karpukhin, Hu Xu, Naman Goyal, Dmytro Okhonko, Mandar Joshi, Gargi Ghosh, Mike Lewis, Luke Zettlemoyer

link-bibliography
2022-liu-2.pdf: “Design Guidelines for Prompt Engineering Text-To-Image Generative Models ”, Vivian Liu, Lydia B. Chilton

link-bibliography
https://arxiv.org/abs/2201.02605#facebook: “Detecting Twenty-Thousand Classes Using Image-Level Supervision ”, Xingyi Zhou, Rohit Girdhar, Armand Joulin, Phillip Krähenbühl, Ishan Misra

link-bibliography
https://arxiv.org/abs/2112.10752: “High-Resolution Image Synthesis With Latent Diffusion Models ”, Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, Björn Ommer

link-bibliography
https://arxiv.org/abs/2112.09106#microsoft: “RegionCLIP: Region-Based Language-Image Pretraining ”, Yiwu Zhong, Jianwei Yang, Pengchuan Zhang, Chunyuan Li, Noel Codella, Liunian Harold Li, Luowei Zhou, Xiyang Dai, Lu Yuan, Yin Li, Jianfeng Gao

link-bibliography
https://arxiv.org/abs/2112.05744: “More Control for Free! Image Synthesis With Semantic Diffusion Guidance ”, Xihui Liu, Dong Huk Park, Samaneh Azadi, Gong Zhang, Arman Chopikyan, Yuxiao Hu, Humphrey Shi, Anna Rohrbach, Trevor Darrell

link-bibliography
https://arxiv.org/abs/2112.05253: “MAGMA—Multimodal Augmentation of Generative Models through Adapter-Based Finetuning ”, Constantin Eichenberg, Sidney Black, Samuel Weinbach, Letitia Parcalabescu, Anette Frank

link-bibliography
https://arxiv.org/abs/2112.01071: “DenseCLIP: Extract Free Dense Labels from CLIP ”, Chong Zhou, Chen Change Loy, Bo Dai

link-bibliography
https://arxiv.org/abs/2112.01573: “FuseDream: Training-Free Text-To-Image Generation With Improved CLIP+GAN Space Optimization ”, Xingchao Liu, Chengyue Gong, Lemeng Wu, Shujian Zhang, Hao Su, Qiang Liu

link-bibliography
https://arxiv.org/abs/2111.11432#microsoft: “Florence: A New Foundation Model for Computer Vision ”, Lu Yuan, Dongdong Chen, Yi-Ling Chen, Noel Codella, Xiyang Dai, Jianfeng Gao, Houdong Hu, Xuedong Huang, Boxin Li, Chunyuan Li, Ce Liu, Mengchen Liu, Zicheng Liu, Yumao Lu, Yu Shi, Lijuan Wang, Jianfeng Wang, Bin Xiao, Zhen Xiao, Jianwei Yang, Michael Zeng, Luowei Zhou, Pengchuan Zhang

link-bibliography
https://arxiv.org/abs/2111.10050#google: “BASIC: Combined Scaling for Open-Vocabulary Image Classification ”, Hieu Pham, Zihang Dai, Golnaz Ghiasi, Kenji Kawaguchi, Hanxiao Liu, Adams Wei Yu, Jiahui Yu, Yi-Ting Chen, Minh-Thang Luong, Yonghui Wu, Mingxing Tan, Quoc V. Le

link-bibliography
https://arxiv.org/abs/2111.09734: “ClipCap: CLIP Prefix for Image Captioning ”, Ron Mokady, Amir Hertz, Amit H. Bermano

link-bibliography
https://arxiv.org/abs/2111.07991#google: “LiT: Zero-Shot Transfer With Locked-Image Text Tuning ”, Xiaohua Zhai, Xiao Wang, Basil Mustafa, Andreas Steiner, Daniel Keysers, Alexander Kolesnikov, Lucas Beyer

link-bibliography
https://arxiv.org/abs/2111.03930: “Tip-Adapter: Training-Free CLIP-Adapter for Better Vision-Language Modeling ”, Renrui Zhang, Rongyao Fang, Peng Gao, Wei Zhang, Kunchang Li, Jifeng Dai, Yu Qiao, Hongsheng Li

link-bibliography
https://arxiv.org/abs/2111.03133: “StyleCLIPDraw: Coupling Content and Style in Text-To-Drawing Synthesis ”, Peter Schaldenbrand, Zhixuan Liu, Jean Oh

link-bibliography
https://arxiv.org/abs/2111.02114#laion: “LAION-400M: Open Dataset of CLIP-Filtered 400 Million Image-Text Pairs ”, Christoph Schuhmann, Richard Vencu, Romain Beaumont, Robert Kaczmarczyk, Clayton Mullis, Aarush Katta, Theo Coombes, Jenia Jitsev, Aran Komatsuzaki

link-bibliography
https://arxiv.org/abs/2111.01007: “Projected GANs Converge Faster ”, Axel Sauer, Kashyap Chitta, Jens Müller, Andreas Geiger

link-bibliography
https://arxiv.org/abs/2110.05208: “Supervision Exists Everywhere: A Data Efficient Contrastive Language-Image Pre-Training Paradigm (DeCLIP) ”, Yangguang Li, Feng Liang, Lichen Zhao, Yufeng Cui, Wanli Ouyang, Jing Shao, Fengwei Yu, Junjie Yan

link-bibliography
https://openreview.net/forum?id=ROteIE-4A6W: “MA-CLIP: Towards Modality-Agnostic Contrastive Language-Image Pre-Training ”, Haoxuan You, Luowei Zhou, Bin Xiao, Noel C. Codella, Yu Cheng, Ruochen Xu, Shih-Fu Chang, Lu Yuan

link-bibliography
https://openreview.net/forum?id=G89-1yZLFHk: “OTTER: Data Efficient Language-Supervised Zero-Shot Recognition With Optimal Transport Distillation ”, Bichen Wu, Ruizhe Cheng, Peizhao Zhang, Peter Vajda, Joseph E. Gonzalez

link-bibliography
https://openreview.net/forum?id=qw674L9PfQE: “CLOOB: Modern Hopfield Networks With InfoLOOB Outperform CLIP ”, Andreas Fürst, Elisabeth Rumetshofer, Viet Tran, Hubert Ramsauer, Fei Tang, Johannes Lehner, David Kreil, Michael Kopp, Günter Klambauer, Angela Bitto-Nemling, Sepp Hochreiter

link-bibliography
https://arxiv.org/abs/2109.12066: “ZSD-YOLO: Zero-Shot YOLO Detection Using Vision-Language Knowledge Distillation ”, Johnathan Xie, Shuai Zheng

link-bibliography
https://www.frontiersin.org/articles/10.3389/fninf.2021.679838/full: “THINGSvision: A Python Toolbox for Streamlining the Extraction of Activations From Deep Neural Networks ”, Lukas Muttenthaler, Martin N. Hebart

link-bibliography
https://arxiv.org/abs/2109.08857#google: “Modern Evolution Strategies for Creativity: Fitting Concrete Images and Abstract Concepts ”, Yingtao Tian, David Ha

link-bibliography
https://arxiv.org/abs/2109.07301: “What Vision-Language Models ‘See’ When They See Scenes ”, Michele Cafagna, Kees van Deemter, Albert Gatt

link-bibliography
https://laion.ai/blog/laion-400-open-dataset/: “LAION-400-Million Open Dataset ”, Christoph Schuhmann

link-bibliography
https://arxiv.org/abs/2107.12518: “Segmentation in Style: Unsupervised Semantic Image Segmentation With StyleGAN and CLIP ”, Daniil Pakhomov, Sanchit Hira, Narayani Wagle, Kemar E. Green, Nassir Navab

link-bibliography
https://arxiv.org/abs/2107.06383: “How Much Can CLIP Benefit Vision-And-Language Tasks? ”, Sheng Shen, Liunian Harold Li, Hao Tan, Mohit Bansal, Anna Rohrbach, Kai-Wei Chang, Zhewei Yao, Kurt Keutzer

link-bibliography
https://arxiv.org/abs/2107.00650: “CLIP-It! Language-Guided Video Summarization ”, Medhini Narasimhan, Anna Rohrbach, Trevor Darrell

link-bibliography
https://arxiv.org/abs/2106.16198: “Small In-Distribution Changes in 3D Perspective and Lighting Fool Both CNNs and Transformers ”, Spandan Madan, Tomotake Sasaki, Tzu-Mao Li, Xavier Boix, Hanspeter Pfister

link-bibliography
https://arxiv.org/abs/2106.13043: “AudioCLIP: Extending CLIP to Image, Text and Audio ”, Andrey Guzhov, Federico Raue, Jörn Hees, Andreas Dengel

link-bibliography
https://arxiv.org/abs/2106.11097: “CLIP2Video: Mastering Video-Text Retrieval via Image CLIP ”, Han Fang, Pengfei Xiong, Luhui Xu, Yu Chen

link-bibliography
https://arxiv.org/abs/2106.07411: “Partial Success in Closing the Gap between Human and Machine Vision ”, Robert Geirhos, Kantharaju Narayanappa, Benjamin Mitzkus, Tizian Thieringer, Matthias Bethge, Felix A. Wichmann, Wiel, Brendel

link-bibliography
https://arxiv.org/abs/2106.03004#google: “Exploring the Limits of Out-Of-Distribution Detection ”, Stanislav Fort, Jie Ren, Balaji Lakshminarayanan

link-bibliography
https://en.pingwest.com/a/8693#baai: “Chinese AI Lab Challenges Google, OpenAI With a Model of 1.75 Trillion Parameters ”, Chen Du

link-bibliography
https://arxiv.org/abs/2104.13921#google: “Zero-Shot Detection via Vision and Language Knowledge Distillation ”, Xiuye Gu, Tsung-Yi Lin, Weicheng Kuo, Yin Cui

link-bibliography
https://arxiv.org/abs/2104.08945#facebook: “Data-Efficient Language-Supervised Zero-Shot Learning With Self-Distillation ”, Ruizhe Cheng, Bichen Wu, Peizhao Zhang, Peter Vajda, Joseph E. Gonzalez

link-bibliography
https://distill.pub/2021/multimodal-neurons/#openai: “Multimodal Neurons in Artificial Neural Networks [CLIP] ”, Gabriel Goh, Nick Cammarata, Chelsea Voss, Shan Carter, Michael Petrov, Ludwig Schubert, Alec Radford, Chris Olah

link-bibliography
https://arxiv.org/abs/2102.05918#google: “ALIGN: Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervision ”, Chao Jia, Yinfei Yang, Ye Xia, Yi-Ting Chen, Zarana Parekh, Hieu Pham, Quoc V. Le, Yunhsuan Sung, Zhen Li, Tom Duerig

link-bibliography
https://arxiv.org/abs/2102.01645: “Generating Images from Caption and vice Versa via CLIP-Guided Generative Latent Space Search ”, Federico A. Galatolo, Mario G. C. A. Cimino, Gigliola Vaglini

link-bibliography
https://github.com/nagolinc/notebooks/blob/main/TADNE_and_CLIP.ipynb: “Scoring Images from TADNE With CLIP ”, nagolinc

link-bibliography
https://cdn.openai.com/papers/Learning_Transferable_Visual_Models_From_Natural_Language_Supervision.pdf: “CLIP: Learning Transferable Visual Models From Natural Language Supervision ”, Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya A. Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, Ilya Sutskever

link-bibliography
https://openai.com/index/clip/: “CLIP: Connecting Text and Images: We’re Introducing a Neural Network Called CLIP Which Efficiently Learns Visual Concepts from Natural Language Supervision. CLIP Can Be Applied to Any Visual Classification Benchmark by Simply Providing the Names of the Visual Categories to Be Recognized, Similar to the ‘Zero-Shot’ Capabilities of GPT-2 and GPT-3 ”, Alec Radford, Ilya Sutskever, Jong Wook Kim, Gretchen Krueger, Sandhini Agarwal

link-bibliography
https://openai.com/index/dall-e/: “DALL·E 1: Creating Images from Text: We’ve Trained a Neural Network Called DALL·E That Creates Images from Text Captions for a Wide Range of Concepts Expressible in Natural Language ”, Aditya A. Ramesh, Mikhail Pavlov, Gabriel Goh, Scott Gray, Mark Chen, Rewon Child, Vedant Misra, Pamela Mishkin, Gretchen Krueger, Sandhini Agarwal, Ilya Sutskever

link-bibliography
https://arxiv.org/abs/2010.11929#google: “Vision Transformer: An Image Is Worth 16×16 Words: Transformers for Image Recognition at Scale ”, Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, Neil Houlsby

link-bibliography
https://www.technologyreview.com/2020/02/17/844721/ai-openai-moonshot-elon-musk-sam-altman-greg-brockman-messy-secretive-reality/: “The Messy, Secretive Reality behind OpenAI’s Bid to save the World: The AI Moonshot Was Founded in the Spirit of Transparency. This Is the inside Story of How Competitive Pressure Eroded That Idealism ”, Karen Hao

link-bibliography
https://github.com/utilityhotbar/sam2_hierarch: “sam2_hierarch: Unsupervised Human-Friendly Online Object Categorization ”, UtilityHotbar

link-bibliography