TODO

[Dongwoo-Im]

Image

• 2024/11 Analyzing The Language of Visual Tokens (ICLR 2025 Submitted)
• 2024/11 ALIT: Adaptive Length Image Tokenization via Recurrent Allocation (ICLR 2025 Submitted)
• 2024/04 Editable Image Elements for Controllable Synthesis (ECCV 2024)
• 2024/01 pix2gestalt: Amodal Segmentation by Synthesizing Wholes (CVPR 2024 Hightlight)
• 2023/12 Amodal Completion via Progressive Mixed Context Diffusion (CVPR 2024 Highlight)

VLM (MLLM)

• 2024/11 Mixture-of-Transformers: A Sparse and Scalable Architecture for Multi-Modal Foundation Models
• 2024/10 γ−MoD: Exploring Mixture-of-Depth Adaptation for Multimodal Large Language Models (ICLR 2025 Submitted)
• 2024/10 CLEAR: Character Unlearning in Textual and Visual Modalities
• 2024/09 MM1.5: Methods, Analysis & Insights from Multimodal LLM Fine-tuning (ICLR 2025 Submitted)
• 2024/04 BLINK: Multimodal Large Language Models Can See but Not Perceive (ECCV 2024)
• 2024/03 MagicLens: Self-Supervised Image Retrieval with Open-Ended Instructions (ICML 2024 Oral)
• 2024/03 Mini-Gemini: Mining the Potential of Multi-modality Vision Language Models
• 2024/03 MyVLM: Personalizing VLMs for User-Specific Queries (ECCV 2024)
• 2024/03 MM1: Methods, Analysis & Insights from Multimodal LLM Pre-training
• 2024/01 LLaVA-NeXT: Improved reasoning, OCR, and world knowledge
• 2023/12 FIND: Interfacing Foundation Models' Embeddings (NIPS 2024)

Mamba

• 2024/10 MatMamba: A Matryoshka State Space Model (ICLR 2025 Submitted)
• 2024/03 Cobra: Extending Mamba to Multi-Modal Large Language Model for Efficient Inference
• 2024/03 ZigMa: Zigzag Mamba Diffusion Model (ECCV 2024)
• 2024/03 VideoMamba: State Space Model for Efficient Video Understanding (ECCV 2024)
• 2024/03 LightM-UNet: Mamba Assists in Lightweight UNet for Medical Image Segmentation

Gen Image

• 2024/11 SVDQunat: Absorbing Outliers by Low-Rank Components for 4-Bit Diffusion Models (ICLR 2025 Submitted)
• 2024/04 Guiding a Diffusion Model with a Bad Version of Itself (NIPS 2024 Best Paper Runner-up Award)
• 2024/04 Paint by Inpaint: Learning to Add Image Objects by Removing Them First
• 2024/04 InstantFamily: Masked Attention for Zero-shot Multi-ID Image Generation
• 2024/04 Stylus: Automatic Adapter Selection for Diffusion Models (NIPS 2024 Oral)
• 2024/04 EdgeFusion: On-Device Text-to-Image Generation (CVPRW 2024)
• 2024/04 UniFL: Improve Stable Diffusion via Unified Feedback Learning (NIPS 2024)
• 2024/04 CAN: Condition-Aware Neural Network for Controlled Image Generation (CVPR 2024)
• 2024/04 VAR: Visual Autoregressive Modeling: Scalable Image Generation via Next-Scale Prediction (NIPS 2024 Best Paper Award)
• 2024/04 T-GATE: Cross-Attention Makes Inference Cumbersome in Text-to-Image Diffusion Models
• 2024/04 Bigger is not Always Better: Scaling Properties of Latent Diffusion Models
• 2024/03 ReNoise: Real Image Inversion Through Iterative Noising (ECCV 2024)
• 2024/02 DistriFusion: Distributed Parallel Inference for High-Resolution Diffusion Models (CVPR 2024)
• 2024/02 Neural Network Diffusion
• 2024/02 InstanceDiffusion: Instance-level Control for Image Generation (CVPR 2024)
• 2024/01 Mastering Text-to-Image Diffusion: Recaptioning, Planning, and Generating with Multimodal LLMs (ICML 2024)
• 2023/09 Finite Scalar Quantization: VQ-VAE Made Simple (ICLR 2024)
• 2023/03 Understanding Diffusion Objectives as the ELBO with Simple Data Augmentation (NIPS 2023 Oral)
• 2022/12 DiT: Scalable Diffusion Models with Transformers (ICCV 2023 Oral)

Video

• 2024/10 MarDini: Masked Autoregressive Diffusion for Video Generation at Scale (ICLR 2025 Submitted)
• 2024/03 DINO-Tracker: Taming DINO for Self-Supervised Point Tracking in a Single Video (ECCV 2024)
• 2024/03 Be-Your-Outpainter: Mastering Video Outpainting through Input-Specific Adaptation (ECCV 2024)
• 2024/03 Mora: More like Sora for Generalist Video Generation
• 2024/02 MC-ViT: Memory Consolidation Enables Long-Context Video Understanding

3D

• 2024/06 Grounding Image Matching in 3D with MASt3R
• 2024/05 SUNDAE: Spectrally Pruned Gaussian Fields with Neural Compensation
• 2024/04 RefFusion: Reference Adapted Diffusion Models for 3D Scene Inpainting
• 2024/03 2DGS: 2D Gaussian Splatting for Geometrically Accurate Radiance Fields (SIGGRAPH 2024)
• 2024/03 RadSplat: Radiance Field-Informed Gaussian Splatting for Robust Real-Time Rendering with 900+ FPS
• 2024/03 MVEdit: Generic 3D Diffusion Adapter Using Controlled Multi-View Editing
• 2023/12 DUSt3R: Geometric 3D Vision Made Easy (CVPR 2024)
• 2023/12 Feature3DGS: Supercharging 3D Gaussian Splatting to Enable Distilled Feature Fields (CVPR 2024 Highlight)
• 2023/12 SMERF: Streamable Memory Efficient Radiance Fields for Real-Time Large-Scene Exploration (SIGGRAPH 2024)

Efficinet LLM

• 2024/11 BitNet a4.8: 4-bit Activations for 1-bit LLMs
• 2024/07 ThinK: Thinner Key Cache by Query-Driven Pruning (ICLR 2025 Submitted)
• 2024/06 What Matters in Transformers? Not All Attention is Needed (ICLR 2025 Submitted)
• 2024/04 Better & Faster Large Language Models via Multi-token Prediction (ICML 2024)
• 2024/04 I-Design: Personalized LLM Interior Designer
• 2024/04 ReFT: Representation Finetuning for Language Models (NIPS 2024 Spotlight)
• 2024/03 Transformer-Lite: High-efficiency Deployment of Large Language Models on Mobile Phone GPUs
• 2024/03 The Unreasonable Ineffectiveness of the Deeper Layers (ICLR 2025 Submitted)
• 2024/03 EMMET: A Unified Framework for Model Editing (EMNLP 2024 Findings)
• 2024/03 PERL: Parameter Efficient Reinforcement Learning from Human Feedback
• 2024/03 GaLore: Memory-Efficient LLM Training by Gradient Low-Rank Projection (ICML 2024 Oral)
• 2024/01 KVQuant: Towards 10 Million Context Length LLM Inference with KV Cache Quantization (NIPS 2024)
• 2023/06 AWQ: Activation-aware Weight Quantization for LLM Compression and Acceleration (MLSys 2024 Best Paper Award)
• 2023/02 DoRA: Weight-Decomposed Low-Rank Adaptation (ICML 2024 Oral)
• 2023/02 Ten Lessons We Have Learned in the New “Sparseland”: A Short Handbook for Sparse Neural Network Researchers (ICLR 2023 Spotlight)
• 2023/01 SparseGPT: Massive Language Models Can Be Accurately Pruned in One-Shot (ICML 2023)

LLM

• 2024/09 Gemma Scope: Open Sparse Autoencoders Everywhere All At Once on Gemma 2 (EMNLPW 2024)
• 2024/04 Rho-1: Not All Tokens Are What You Need (NIPS 2024 Best Paper Runner-up Award)
• 2024/04 Iterative RPO: Iterative Reasoning Preference Optimization (NIPS 2024)
• 2024/04 Layer Skip: Enabling Early Exit Inference and Self-Speculative Decoding (ACL 2024)
• 2024/04 Mixture-of-Depths: Dynamically allocating compute in transformer-based language models
• 2024/03 Gecko: Versatile Text Embeddings Distilled from Large Language Models
• 2024/03 RAFT: Adapting Language Model to Domain Specific RAG (COLM 2024)
• 2024/02 Echo Embeddings: Repetition Improves Language Model Embeddings (ICLR 2025 Submitted)
• 2023/12 The Unlocking Spell on Base LLMs: Rethinking Alignment via In-Context Learning (ICLR 2024)
• 2023/10 Controlled Decoding from Language Models (ICML 2024)
• 2023/10 Demystifying Embedding Spaces using Large Language Models (ICLR 2024)
• 2023/08 Linearity of Relation Decoding in Transformer Language Models (ICLR 2024 Spotlight)

ETC

• 2024/03 FaceXFormer : A Unified Transformer for Facial Analysis
• 2024/03 DiPaCo: Distributed Path Composition
• 2024/02 Genie: Generative Interactive Environments (ICML 2024 Best Paper Award)
• 2024/01 Depth Anything: Unleashing the Power of Large-Scale Unlabeled Data (CVPR 2024)
• 2023/12 FuseMix: Data-Efficient Multimodal Fusion on a Single GPU (CVPR 2024 Highlight)

Good Article

• https://www.github.com/pytorch-labs/ao/issues/47
• https://deepmind.google/discover/blog/sima-generalist-ai-agent-for-3d-virtual-environments/
• https://pytorch.org/blog/accelerating-generative-ai-2/
• https://github.com/microsoft/unilm/tree/master/bitnet
• https://huggingface.co/blog/embedding-quantization
• https://timdettmers.com/2023/01/30/which-gpu-for-deep-learning/
• https://www.ai21.com/blog/announcing-jamba
• https://sambanova.ai/blog/accurate-models-at-blazing-speed
• https://aws.amazon.com/ko/blogs/tech/klleon-digital-human-generative-model-aws-inferentia/
• https://github.com/google-research-datasets
• https://carpedm30.notion.site/AI-Compiler-Study-aaf4cff2c8734e50ad95ac6230dbd80b
• https://deepmind.google/discover/blog/tacticai-ai-assistant-for-football-tactics/
• https://huggingface.co/docs/transformers/v4.15.0/parallelism
• Google, ScaNN (Scalable Nearest Neighbors)
• https://moon-walker.medium.com/%EB%A6%AC%EB%B7%B0-zero-infinity-breaking-the-gpu-memory-wall-for-extreme-scale-deep-learning-eeca3c035da5
• https://www.microsoft.com/en-us/research/blog/deepspeed-extreme-scale-model-training-for-everyone/
• https://thegenerality.com/agi