Drawing Animals in Motion Collection

Animal Paper Collection (Ongoing)

[2021.11.28] I recommend MMPose for 2D animal pose estimation. It has collected various kinds of datasets.

[2021.07.16] I recommend CV4Animal workshop in CVPR 2021 (https://www.cv4animals.com/) because it is a good collection of recent advances in animal pose estimation area!

Recently, markerless animal motion capture and 3D reconstruction attracts more and more attention in computer vision community. Inspired by remarkable techniques for markerless human motion capture, a few excellent literatures appear for animal modeling and reconstruction such as SMAL and DeepLabCut. However, there is still a long way before computer vision methods could capture natural motion of arbitary animals in industrial-grade.

Therefore, I contribute this repository to track every step towards the ultimate goal of high quality animal capture. If you want to add/remove an article, please send an email to Liang An(al17 at mails dot tsinghua dot edu dot cn). Thank all the authors for their contribution and support.

The Mouse Action Recognition System (MARS) software pipeline for automated analysis of social behaviors in mice
Cristina Segalin, Jalani Williams, Tomomi Karigo, May Hui, Moriel Zelikowsky, Jennifer J Sun, Pietro Perona, David J Anderson, Ann Kennedy
In eLife 2021
[Paper] [Project]
Hierarchical organization of rhesus macaque behavior
Benjamin Voloh, Benjamin R. Eisenreich, David J-N. Maisson, R. Becket Ebitz, Hyun Soo Park, Benjamin Y. Hayden, Jan Zimmermann
In biorxiv 2021
[Paper]
NeuroMechFly, a neuromechanical model of adult Drosophila melanogaster
Victor Lobato Rios, Pembe Gizem Ozdil, Shravan Tata Ramalingasetty, Jonathan Arreguit, Auke Jan Ijspeert, Pavan Ramdya
In biorxiv 2021
[Paper] [Project]
OpenMonkeyChallenge: Dataset and Benchmark Challenges for Pose Tracking of Non-human Primates
Yuan Yao, Abhiraj Mohan, Eliza Bliss-Moreau, Kristine Coleman, Sienna M. Freeman, Christopher J. Machado, Jessica Raper, Jan Zimmermann, Benjamin Y. Hayden, Hyun Soo Park
In biorxiv 2021
[Paper] [Project]
ThruTracker: Open-Source Software for 2-D and 3-D Animal Video Tracking
Aaron J. Corcoran, Michael R. Schirmacher, Eric Black, Tyson L. Hedrick
In biorxiv 2021
[Paper]
A Novel Dataset for Keypoint Detection of Quadruped Animals from Images
Prianka Banik, Lin Li, Xishuang Dong
In arxiv 2021
[Paper]
Leaving Flatland: Advances in 3D behavioral measurement
Jesse D. Marshall, Tianqing Li, Joshua H. Wu, Timothy W. Dunn
In arxiv 2021
[Paper]
Multimodal-based Scene-Aware Framework for Aquatic Animal Segmentation
Minh-Quan Le, Trung-Nghia Le, Tam V. Nguyen, Isao Echizen, Minh-Triet Tran
In arxiv 2021
[Paper]
Self-Supervised Keypoint Discovery in Behavioral Videos
Jennifer J. Sun, Serim Ryou, Roni Goldshmid, Brandon Weissbourd, John Dabiri, David J. Anderson, Ann Kennedy, Yisong Yue, Pietro Perona
In arxiv 2021
[Paper]
Big behavior: challenges and opportunities in a new era of deep behavior profiling
Lukas von Ziegler, Oliver Sturman, Johannes Bohacek
In Neuropsychopharmacology 2021
[Paper]
The Multi-Agent Behavior Dataset: Mouse Dyadic Social Interactions
Jennifer J. Sun, Tomomi Karigo, Dipam Chakraborty, Sharada P. Mohanty, David J. Anderson, Pietro Perona, Yisong Yue, Ann Kennedy
In NeurIPS (Dataset & Benchmarks) 2021
[Paper] [Project]
AP-10K: A Benchmark for Animal Pose Estimation in the Wild
Hang Yu, Yufei Xu, Jing Zhang, Wei Zhao, Ziyu Guan, Dacheng Tao
In NeurIPS 2021
[Paper] [Project]
Across-animal odor decoding by probabilistic manifold alignment
Pedro Herrero-Vidal, Dmitry Rinberg, Cristina Savin
In NeurIPS 2021
[Paper]
Coarse-to-fine Animal Pose and Shape Estimation
Chen Li, Gim Hee Lee
In NeurIPS 2021
[Paper]
Learning interaction rules from multi-animal trajectories via augmented behavioral models
Keisuke Fujii, Naoya Takeishi, Kazushi Tsutsui, Emyo Fujioka, Nozomi Nishiumi, Ryoya Tanaka, Mika Fukushiro, Kaoru Ide, Hiroyoshi Kohno, Ken Yoda, Susumu Takahashi, Shizuko Hiryu, Yoshinobu Kawahara
In NeurIPS 2021
[Paper]
The PAIR-R24M Dataset for Multi-animal 3D Pose Estimation
Jesse Marshall, Ugne Klibaite, Amanda Gellis, Diego Aldarondo, Bence O¨lveczky, Tim Dunn
In NeurIPS 2021
[Paper] [Project]
DeepImageJ: A user-friendly environment to run deep learning models in ImageJ
Estibaliz Gómez-de-Mariscal, Carlos García-López-de-Haro, Wei Ouyang, Laurène Donati, Emma Lundberg, Michael Unser, Arrate Muñoz-Barrutia, Daniel Sage
In Nature Methods (brief communication) 2021
[Paper] [Project]
A three-dimensional virtual mouse generates synthetic training data for behavioral analysis
Luis A. Bolaños, Dongsheng Xiao, Nancy L. Ford, Jeff M. LeDue, Pankaj K. Gupta, Carlos Doebeli, Hao Hu, Helge Rhodin, Timothy H. Murphy
In Nature Methods (Brief Communication) 2021 (cover)
[Paper] [Project]
Geometric deep learning enables 3D kinematic profiling across species and environments
Timothy W. Dunn, Jesse D. Marshall, Kyle S. Severson, Diego E. Aldarondo, David G. C. Hildebrand, Selmaan N. Chettih, William L. Wang, Amanda J. Gellis, David E. Carlson, Dmitriy Aronov, Winrich A. Freiwald, Fan Wang, Bence P. Ölveczky
In Nature Methods 2021
[Paper] [Project]
LiftPose3D, a deep learning-based approach for transforming two-dimensional to three-dimensional poses in laboratory animals
Adam Gosztolai, Semih Gunel, Marco Pietro Abrate, Daniel Morales, Victor Lobato Rios, Helge Rhodin, Pascal Fua, Pavan Ramdya
In Nature Methods 2021
[Paper] [Project]
A Hierarchical 3D-motion Learning Framework for Animal Spontaneous Behavior Mapping
Kang Huang, Yaning Han, Ke Chen, Hongli Pan, Wenling Yi, Xiaoxi Li, Siyuan Liu, Liping Wang, Pengfei Wei
In Nature Communications 2021
[Paper] [Project]
4DComplete: Non-Rigid Motion Estimation Beyond the Observable Surface
Yang Li, Hikari Takehara, Takafumi Taketomi, Bo Zheng, Matthias Nießner
In ICCV 2021
[Paper] [Project]
DensePose 3D: Lifting Canonical Surface Maps of Articulated Objects to the Third Dimension
Roman Shapovalov, David Novotny, Benjamin Graham, Patrick Labatut, Andrea Vedaldi
In ICCV 2021
[Paper]
Body language signals for rodent social communication
Christian L. Ebbesen, Robert C. Froemke
In Current Opinion in Neurobiology 2021
[Paper]
DeepLabStream enables closed-loop behavioral experiments using deep learning-based markerless, real-time posture detection
Jens F. Schweihoff, Matvey Loshakov, Irina Pavlova, Laura Kück, Laura A. Ewell, Martin K. Schwarz
In Communications Biology 2021
[Paper] [Project]
Anipose: a toolkit for robust markerless 3D pose estimation
Pierre Karashchuk, Katie L. Rupp, Evyn S. Dickinson, SarahWalling-Bell, Elischa Sanders, Eiman Azim, Bingni W. Brunton, John C. Tuthill
In Cell Reports (Resource) 2021
[Paper] [Project]
Birds of a Feather: Capturing Avian Shape Models from Images
Yufu Wang, Nikos Kolotouros, Kostas Daniilidis, Marc Badger
In CVPR 2021
[Paper] [Project]
Discovering Relationships between Object Categories via Universal Canonical Maps
Natalia Neverova, Artsiom Sanakoyeu, Patrick Labatut, David Novotny, Andrea Vedaldi
In CVPR 2021
[Paper]
From Synthetic to Real: Unsupervised Domain Adaptation for Animal Pose Estimation
Chen Li, Gim Hee Lee
In CVPR 2021 (oral)
[Paper] [Project]
LASR: Learning Articulated Shape Reconstruction from a Monocular Video
Gengshan Yang, Deqing Sun, Varun Jampani, Daniel Vlasic, Forrester Cole, Huiwen Chang, Deva Ramanan, William T. Freeman, Ce Liu
In CVPR 2021
[Paper] [Project]
Task Programming: Learning Data Efficient Behavior Representations
Jennifer J. Sun, Ann Kennedy, Eric Zhan, David J. Anderson, Yisong Yue, Pietro Perona
In CVPR 2021 (Best Student Paper Award)
[Paper] [Project]
Unified 3D Mesh Recovery of Humans and Animals by Learning Animal Exercise
Kim Youwang, Kim Ji-Yeon, Kyungdon Joo, Tae-Hyun Oh
In BMVC 2021
[Paper]
Dynamics of gaze control during prey capture in freely moving mice
Angie M Michaiel, Elliott TT Abe, Cristopher M Niell
In eLife 2020
[Paper]
Real-time, low-latency closed-loop feedback using markerless posture tracking
Gary Kane, Gonçalo Lopes, Jonny L. Saunders, Alexander Mathis, Mackenzie W. Mathis
In eLife 2020 (Featured by Nature Methods)
[Paper] [Project]
AlphaTracker: A Multi-Animal Tracking and Behavioral Analysis Tool
Zexin Chen, Ruihan Zhang, Yu Eva Zhang, Haowen Zhou, Hao-Shu Fang, Rachel R. Rock, Aneesh Bal, Nancy Padilla-Coreano, Laurel Keyes, Kay M. Tye, Cewu Lu
In biorxiv 2020
[Paper] [Project]
Automatic mapping of multiplexed social receptive fields by deep learning and GPU-accelerated 3D videography
Christian L. Ebbesen, Robert C. Froemke
In biorxiv 2020
[Paper]
FreiPose: A Deep Learning Framework for Precise Animal Motion Capture in 3D Spaces
Christian Zimmermann, Artur Schneider, Mansour Alyahyay, Thomas Brox, Ilka Diester
In biorxiv 2020
[Paper] [Project]
Gait-level analysis of mouse open field behavior using deep learning-based pose estimation
Keith Sheppard, Justin Gardin, Gautam Sabnis, Asaf Peer, Megan Darrell, Sean Deats, Brian Geuther, Cathleen M. Lutz, Vivek Kumar
In biorxiv 2020
[Paper]
SLEAP: Multi-animal pose tracking
Talmo D. Pereira, Nathaniel Tabris, Junyu Li, Shruthi Ravindranath, Eleni S. Papadoyannis, Z. Yan Wang, David M. Turner, Grace McKenzie-Smith, Sarah D. Kocher, Annegret L. Falkner, Joshua W. Shaevitz, Mala Murthy
In biorxiv 2020
[Paper] [Project]
Simple Behavioral Analysis (SimBA)-an open source toolkit for computer classification of complex social behaviors in experimental animals
Simon RO Nilsson, Nastacia L. Goodwin, Jia Jie Choong, Sophia Hwang, Hayden R Wright, Zane C Norville, Xiaoyu Tong, Dayu Lin, Brandon S. Bentzley, Neir Eshel, Ryan J McLaughlin, Sam A. Golden
In biorxiv 2020
[Paper]
TRex, a fast multi-animal tracking system with markerless identification, 2D body posture estimation and visual field reconstruction
Tristan Walter, Iain D Couzin
In biorxiv 2020
[Paper]
Muti-view Mouse Social Behaviour Recognition with Deep Graphic Model
Zheheng Jiang, Feixiang Zhou, Aite Zhao, Xin Li, Ling Li, Dacheng Tao, Xuelong Li, Huiyu Zhou
In arxiv 2020
[Paper]
Multiview Supervision By Registration
Yilun Zhang, Hyun Soo Park
In WACV 2020
[Paper]
Facial expressions of emotion states and their neuronal correlates in mice
Nejc Dolensek, Daniel A. Gehrlach, Alexandra S. Klein, Nadine Gogolla
In Science 2020
[Paper]
ZooBuilder: 2D and 3D Pose Estimation for Quadrupeds Using Synthetic Data
Abassin Sourou Fangbemi, Yi Fei Lu, Mao Yuan Xu, Xiao Wu Luo, Alexis Rolland, Chedy Raissi
In SCA (ACM Siggraph/Eurographics Symposium on Computer Animation) 2020
[Paper]
A Primer on Motion Capture with Deep Learning: Principles, Pitfalls and Perspectives
Alexander Mathis, Steffen Schneider, Jessy Lauer, Mackenzie W. Mathis
In Neuron 2020
[Paper]
Continuous Whole-Body 3D Kinematic Recordings across the Rodent Behavioral Repertoire
Jesse D. Marshall, Diego E. Aldarondo, Timothy W. Dunn, William L. Wang, Gordon J. Berman, Bence P. O¨lveczky
In Neuron 2020
[Paper]
Deep Graph Pose: a semi-supervised deep graphical model for improved animal pose tracking
Anqi Wu, E. Kelly Buchanan, Matthew Whiteway, Michael Schartner, Guido Meijer, Jean-Paul Noel, Erica Rodriguez, Claire Everett, Amy Norovich, Evan Schaffer, Neeli Mishra, C. Daniel Salzman, Dora Angelaki, Andrés Bendesky, The International Brain Laboratory, John Cunningham, Liam Paninski
In NeurIPS 2020
[Paper] [Project]
Quantifying behavior to understand the brain
Talmo D. Pereira, Joshua W. Shaevitz, Mala Murthy
In Nature Neuroscience 2020 (review article)
[Paper]
EthoLoop: automated closed-loop neuroethology in naturalistic environments
Ali Nourizonoz, Robert Zimmermann, Chun Lum Andy Ho, Sebastien Pellat, Yannick Ormen, Clément Prévost-Solié, Gilles Reymond, Fabien Pifferi, Fabienne Aujard, Anthony Herrel, Daniel Huber
In Nature Methods 2020 (cover)
[Paper]
Automated Markerless Pose Estimation in Freely Moving Macaques using OpenMonkeyStudio
Praneet C. Bala, Benjamin R. Eisenreich, Seng Bum Michael Yoo, Benjamin Y. Hayden, Hyun Soo Park, Jan Zimmermann
In Nature Communications 2020
[Paper] [Project]
3D Bird Reconstruction: a Dataset, Model, and Shape Recovery from a Single View
Marc Badger, Yufu Wang, Adarsh Modh, Ammon Perkes, Nikos Kolotouros, Bernd Pfrommer, Marc F. Schmidt, Kostas Daniilidis
In ECCV 2020
[Paper] [Project]
Deep Cross-species Feature Learning for Animal Face Recognition via Residual Interspecies Equivariant Network
Xiao Shi, Chenxue Yang, Xue Xia, Xiujuan Chai
In ECCV 2020
[Paper]
Shape and Viewpoint without Keypoints
Shubham Goel, Angjoo Kanazawa, Jitendra Malik
In ECCV 2020
[Paper] [Project]
Who Left the Dogs Out? 3D Animal Reconstruction with Expectation Maximization in the Loop
Benjamin Biggs, Oliver Boyne, James Charles, Andrew Fitzgibbon, Roberto Cipolla
In ECCV 2020
[Paper]
3D-ZeF: A 3D Zebrafish Tracking Benchmark Dataset
Malte Pedersen, Joakim Bruslund Haurum, Stefan Hein Bengtson, Thomas B. Moeslund
In CVPR 2020
[Paper] [Project]
AnimalWeb: A Large-Scale Hierarchical Dataset of Annotated Animal Faces
Muhammad Haris Khan, John McDonagh, Salman Khan, Muhammad Shahabuddin, Aditya Arora, Fahad Shahbaz Khan, Ling Shao, Georgios Tzimiropoulos
In CVPR 2020
[Paper] [Project]
Articulation Aware Canonical Surface Mapping
Nilesh Kulkarni, Abhinav Gupta, David F. Fouhey, Shubham Tulsiani
In CVPR 2020
[Paper] [Project]
Deformation-aware Unpaired Image Translation for Pose Estimation on Laboratory Animals
Siyuan Li, Semih Gunel, Mirela Ostrek, Pavan Ramdya, Pascal Fua, Helge Rhodin
In CVPR 2020
[Paper] [Project]
Learning from Synthetic Animals
Jiteng Mu, Weichao Qiu, Gregory Hager, Alan Yuille
In CVPR 2020 (oral)
[Paper] [Project]
RGBD-Dog: Predicting Canine Pose from RGBD Sensors
Sinead Kearney, Wenbin Li, Martin Parsons, Kwang In Kim, Darren Cosker
In CVPR 2020
[Paper] [Project]
Transferring Dense Pose to Proximal Animal Classes
Artsiom Sanakoyeu, Vasil Khalidov, Maureen S. McCarthy, Andrea Vedaldi, Natalia Neverova
In CVPR 2020
[Paper] [Project]
Unsupervised Learning of Probably Symmetric Deformable 3D Objects from Images in the Wild
Shangzhe Wu, Christian Rupprecht, Andrea Vedaldi
In CVPR 2020 (Best Paper Award)
[Paper] [Project]
DeepPoseKit, a software toolkit for fast and robust animal pose estimation using deep learning
Jacob M. Graving, Daniel Chae, Hemal Naik, Liang Li, Benjamin Koger, Blair R. Costelloe, Iain D. Couzin
In eLife 2019
[Paper] [Project]
Using DeepLabCut for 3D markerless pose estimation across species and behaviors
Tanmay Nath, Alexander Mathis, An Chi Chen, Amir Patel, Matthias Bethge, Mackenzie Weygandt Mathis
In Nature Protocols 2019 (cover)
[Paper] [Project]
idtracker.ai: tracking all individuals in small or large collectives of unmarked animals
Francisco Romero-Ferrero, Mattia G. Bergomi, Robert C. Hinz, Francisco J. H. Heras, Gonzalo G. de Polavieja
In Nature Methods (Brief Communication) 2019
[Paper] [Project]
Fast animal pose estimation using deep neural networks
Talmo D. Pereira, Diego E. Aldarondo, Lindsay Willmore, Mikhail Kislin, Samuel S.-H. Wang, Mala Murthy, Joshua W. Shaevitz
In Nature Methods 2019
[Paper] [Project]
Real-time analysis of the behaviour of groups of mice via a depth-sensing camera and machine learning
Fabrice de Chaumont, Elodie Ey, Nicolas Torquet, Thibault Lagache, Stéphane Dallongeville, Albane Imbert, Thierry Legou, Anne-Marie Le Sourd, Philippe Faure, Thomas Bourgeron, Jean-Christophe Olivo-Marin
In Nature Biomedical Engineering 2019
[Paper] [Project]
Canonical Surface Mapping via Geometric Cycle Consistency
Nilesh Kulkarni, Abhinav Gupta, Shubham Tulsiani
In ICCV 2019
[Paper] [Project]
Cross-Domain Adaptation for Animal Pose Estimation
Jinkun Cao, Hongyang Tang, Hao-Shu Fang, Xiaoyong Shen, Cewu Lu, Yu-Wing Tai
In ICCV 2019 (oral)
[Paper] [Project]
MONET: Multiview Semi-supervised Keypoint Detection via Epipolar Divergence
Yuan Yao, Yasamin Jafarian, Hyun Soo Park
In ICCV 2019
[Paper]
Three-D Safari: Learning to Estimate Zebra Pose, Shape, and Texture from Images "In the Wild"
Silvia Zuffi, Angjoo Kanazawa, Tanya Berger-Wolf, Michael J. Black
In ICCV 2019
[Paper] [Project]
Robust mouse tracking in complex environments using neural networks
Brian Q. Geuther, Sean P. Deats, Kai J. Fox, Steve A. Murray, Robert E. Braun, Jacqueline K. White, Elissa J. Chesler, Cathleen M. Lutz, Vivek Kumar
In Communications Biology 2019
[Paper]
DeepLabCut: markerless pose estimation of user-defined body parts with deep learning
Alexander Mathis, Pranav Mamidanna, Kevin M. Cury, Taiga Abe, Venkatesh N. Murthy, Mackenzie Weygandt Mathis, Matthias Bethge
In Nature Neuroscience 2018
[Paper] [Project]
Learning Category-Specific Mesh Reconstruction from Image Collections
Angjoo Kanazawa, Shubham Tulsiani, Alexei A. Efros, Jitendra Malik
In ECCV 2018
[Paper] [Project]
Lions and Tigers and Bears: Capturing Non-Rigid, 3D, Articulated Shape from Images
Silvia Zuffi, Angjoo Kanazawa, Michael J. Black
In CVPR 2018 (spotlight)
[Paper] [Project]
Creatures great and SMAL: Recovering the shape and motion of animals from video
Benjamin Biggs, Thomas Roddick, Andrew Fitzgibbon, Roberto Cipolla
In ACCV 2018 (oral)
[Paper]
An unsupervised method for quantifying the behavior of paired animals
Ugne Klibaite, Gordon J Berman, Jessica Cande, David L Stern, Joshua W Shaevitz
In Physical Biology 2017
[Paper]
Single-View 3D Reconstruction of Animals
Angjoo Kanazawa
In Ph.D Thesis 2017
[Paper]
3D Menagerie: Modeling the 3D shape and pose of animals
Silvia Zuffi, Angjoo Kanazawa, David W. Jacobs, Michael J. Black
In CVPR 2017 (spotlight)
[Paper] [Project]
3D Pose Estimation of Flying Animals in Multi-view Video Datasets
Mikhail Breslav
In Ph.D Thesis 2016
[Paper]
Learning 3D Deformation of Animals from 2D Images
Angjoo Kanazawa, Shahar Kovalsky, Ronen Basri, David W. Jacobs
In Eurographics 2016 (Günter Enderle Best Paper Award)
[Paper] [Project]
idTracker: Tracking individuals in a group by automatic identification of unmarked animals
Alfonso Pérez-Escudero, Julián Vicente-Page, Robert C Hinz, Sara Arganda, Gonzalo G de Polavieja1
In Nature Methods 2014
[Paper] [Project]
Multi-camera real-time threedimensional tracking of multiple flying animals
Andrew D. Straw, Kristin Branson, Titus R. Neumann, Michael H. Dickinson
In J. R. Soc. Interface 2011
[Paper]
coding of facial expressions of pain in the laboratory mouse
Dale J Langford, Andrea L Bailey, Mona Lisa Chanda, Sarah E Clarke, Tanya E Drummond, Stephanie Echols, Sarah Glick, Joelle Ingrao, Tammy Klassen-Ross, Michael L LaCroix-Fralish, Lynn Matsumiya, Robert E Sorge, Susana G Sotocinal, John M Tabaka, David Wong, Arn M J M van den Maagdenberg, Michel D Ferrari, Kenneth D Craig, Jeffrey S Mogil
In Nature Methods (brief communications) 2010
[Paper]

Last updated in Dec 2021

This repository is inspired by Cat Papers with some code borrowed from it. Thank Junyan Zhu for his sharing!

Drawing Animals in Motion Collection

Source: https://github.com/anl13/animal_papers

0 Response to "Drawing Animals in Motion Collection"

Postar um comentário

Iklan Atas Artikel

Iklan Tengah Artikel 1

Iklan Tengah Artikel 2

Iklan Bawah Artikel