- (badly paraphrased) Einstein.
His research involves designing and developing AI architectures and ML systems for real-time embodied improvisation between non-experts and virtual agents. These systems combine bottom-up interactive action learning with top-down formalizations of improvisational reasoning strategies alongside computational models for evaluating human and agent creativity. This research has been applied to tasks like object substitution reasoning in pretend object play, interactive learning of agent knowledge for improvising dance, and creative decision-making during prop-based improv theatre.
His research has emphasized practice-based methods for understanding how interactive installations featuring improvisational agents can be created. This research also explores how agent architecture designs within public installations impact non-expert user experiences. These installations have been exhibited by invitation and peer-review in venues such as the Smithsonian Institution National Museum of American History (Washington, DC), the ArtScience Museum (Singapore), and the Biltmore (Atlanta).
The EarSketch Co-creative AI research project is exploring how autonomous agents can collaborate with students (potentially as peers) on the online Earsketch platform to increase student exploration of ideas, both musically and computationally.
Currently, researching, designing, and implementing a recommendation engine for musical samples in Earsketch for students to increase the diversity, novelty, and serendipity of sample usage.
The Robot Improv Circus is a VR installation where non-expert users can play the Props game, i.e. improvise open-ended movement-based vignettes with a virtual character using abstract props.
The CARNIVAL (Creative Arc Negotiating Intelligent Virtual Agent pLatform) architecture enables the virtual character, improvising with the human in VR, to select actions along a given ‘creative arc’ over the course of the improvised performance evolving the user’s experience over time.
Currently, researching, designing, and implementing the CARNIVAL intelligent agent architecture that performs affordance-based deep neural action generation; improvisational reasoning using various reasoning strategies; real-time strategy selection to follow a given creative arc; and evaluation of agent/human creativity using computational models of novelty, unexpectedness, and quality.
The research was awarded a Creative Curricular Initiatives (CCI) grant, completed two Invited installations, was featured on the cover of the Georgia Institute of Technology Annual Report 2018, and produced three peer-reviewed publications.
The TuneTable is a tangible computing museum installation for informal learning of computational thinking concepts using open-ended sample-based music composition.
Previously, researched, designed, and constructed two iterations of the interactive tabletop using computer vision.
Two iterations of TuneTable were installed for field experiments at the Museum of Science and Industry, Chicago with peer reviewed results pending.
LuminAI (formerly Viewpoints AI) was an installation that explored how to create a truly open-ended human-AI improvisational embodied interaction experience with minimal pre-authored content knowledge for the AI character.
The system used interactive learning techniques, reasoning strategies from human improvisers, and the Viewpoints framework from theatre & dance were used to learn, procedurally represent, and reason about movement and improvise contemporary movement and dance performances with non-expert users.
Previously, researched, designed, and implemented case-based and imitation learning methods to teach the agent movement improvisation through observation and interaction, as well as procedural reasoning about improvisational response generation within the Soar cognitive architecture.
The research was a Field Experiment grant finalist, successfully completed a collaboration with the T Lang Dance Company (Atlanta) as a hybrid improvised-choreographed dance performance piece called Post, was selected to the ACCelerate Festival at the Smithsonian Institution National Museum of American History, completed numerous international (and domestic) invited and peer-reviewed installations, was the winner of the Neukom Institute Turing Test in Creative Arts 2017: DanceX Prize, and produced six peer-reviewed publications.
The Computational Representations of Play project studied human cognition during pretend play and used the results to computationally model playful behavior in software agents and robots for playful task execution and pretend play with toys.
Previously, researched, designed, and implemented conceptual blending with objects for object-based pretense within a toy-based pretend play experience between human and robot (or embodied virtual agent).
The project included the design of a conceptual cognitive architecture called the Co-creative Cognitive Architecture (CoCoA) extending Soar with an emphasis on co-creativity and real-time improvisation.
The research produced three peer-reviewed publications
The Digital Improv Project was part of an effort to understand improvisational cognition, enabling humans and agents to co-creatively perform improvisational theatre.
Researched computational reasoning about status & representation of character status in improv theatre
Researched, designed, and implemented workflow / process for using crowd-sourcing to collect data forconstructing cognitive scripts for improv theatre.
The Game Adaptive Intelligent Agents (GAIA) project was an agent design system that created agents containing knowledge-based models of their own behavior in order localize the source of reasoning failures, adapt its reasoning to the current situation, and to enable rapid agent prototyping.
Previously, researched new adaptations to the agent design domain (Tic Tac Toe) and implemented models for example game playing agents through domain variations (Miseré TicTacToe and Drawbridge).
1. Jacob, M. & Magerko, B. (2018). “Creative Arcs in Improvised Human Computer Embodied Performances.” In the Proceedings of the 1st Curiosity in Games Workshop at the International Conference on the Foundations of Digital Games (FDG) 2018. Malmö, Sweden. pdf
2. Jacob, M. (2017). “Towards Lifelong Interactive Learning for Open-ended Embodied Co-creative Narrative Improvisation.” In the Proceedings of the Doctoral Consortium at the Eighth International Conference on Computational Creativity (ICCC) 2017, Atlanta, GA USA. pdf
3. Long, D., Jacob, M., Davis, N., and Magerko, B. (2017). “Designing for Socially Interactive Systems.” In the Proceedings of the 11th Conference on Creativity and Cognition (C&C) 2017, Singapore. pdf
4. Jacob, M. (2017). “Towards Lifelong Interactive Learning for Open-ended Embodied Narrative Improvisation.” In the Proceedings of the Graduate Student Symposium at the 11th Conference on Creativity and Cognition (C&C) 2017, Singapore. pdf
5. Singh, K.Y., Davis, N., Hsiao, C.-P., Jacob, M., Patel, K., Magerko, B. (2016). “Recognizing Actions in Motion Trajectories using Deep Neural Networks.” In the Proceedings of the 12th Conference on Artificial Intelligence and Interactive Digital Entertainment (AIIDE) 2016, Burlingame, CA USA. pdf
6. Jacob, M. & Magerko, B. (2015). “Interaction-based Authoring for Scalable Co-creative Agents.” In the Proceedings of the 6th International Conference on Computational Creativity (ICCC) 2015, Provo, UT. pdf
7. Davis, N., Comerford, M., Jacob, M., Hsiao, C.-P., & Magerko, B. (2015). “An Enactive Characterization of Pretend Play.” In the Proceedings of the 10th ACM conference on Creativity and Cognition (C&C) 2015. Glasgow. pdf
8. Jacob, M. & Magerko, B. (2015). “Viewpoints AI.” In the Proceedings of the Artwork Exhibition at the 10th ACM conference on Creativity and Cognition (C&C) 2015. Glasgow, Scotland. pdf
9. Magerko B., Permar, J., Jacob, M., Comerford, M., and Smith, J. (2014). “An Overview of Computational Co-creative Pretend Play with a Human.” In the Proceedings of the 1st Workshop on Playful Virtual Characters at the 14th Annual Conference on Intelligent Virtual Agents (IVA) 2014, Boston, MA USA. pdf
10. Jacob, M., Coisne, G., Gupta, A., Sysoev, I., Verma, G., and Magerko, B. (2013). “Viewpoints AI.” In the Proceedings of the 9th Annual AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment (AIIDE) 2013, Boston, MA USA. pdf
11. Jacob, M., Coisne, G., Gupta, A., Sysoev, I., Verma, G., and Magerko, B. (2013). “Viewpoints AI: Demonstration.” In the Proceedings of the 9th Annual AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment (AIIDE) 2013, Boston, MA USA. pdf
12. Jacob, M., Zook, A., and Magerko, B. (2013). “Viewpoints AI: Procedural Representation and Reasoning on Gesture Meaning.” In the Proceedings of the Digital Games Research Association (DiGRA) 2013, Atlanta, GA USA. pdf
Seeking urgent participants for fall 2018/spring 2019 experiments in playing improv theatre games in virtual reality (VR) with an artificially intelligent (AI) collaborator. Subjects will be paid 15$ for approximately an hour of participation.
Looking for urgent participation from subjects in the Atlanta area. Seeking urgent set of participants for fall 2018/spring 2019 experiments in playing improv theatre games in virtual reality (VR) with an artificially intelligent (AI) collaborator. Subjects will be paid 15$ for approximately an hour of participation.
Post, the fourth performance in the Post Up series, from T Lang Dance Company in collaboration with the ADAM Lab, will feature the current version of Viewpoints AI, titled LuminAI. The performance embodies the powerful emotions and consequences of reuniting with a lost love. Post will be situated inside LuminAI and feature the visual, sonic, and collaborative capabilities of the installation. LuminAI itself is the current version of Viewpoints AI, where humans and AI can create movement-based improvised performances with each other inside a giant geodesic dome, creating a truly immersive human – AI play space. Post has also been selected as a finalist in the Field Experiment grant from the Goat Farm and the Hambidge Center in Atlanta. A sample of Post (and LuminAI) will be premiered at the Hambidge Art Auction event, located at the Goat Farm and held on 23rd April, 2016. It will remain on exhibition at the Goat Farm during evening hours, till 30th April, 2016. The final performance is coming soon, this November 2016.
Details here.
We're urgently seeking participants for a research study to evaluate an interactive art installation called Viewpoints AI. Viewpoints AI is a co-creative human - AI dance / contemporary movement improvisation system, where participants will dance together with the virtual partner agent on screen and evaluate it.
Details here.
85 5th Street,
#325A,
Atlanta, GA. 30308
mikhail<\dot\>jacob<\at\>gatech<\dot\>edu
