This talk will describe recent progress on modeling, planning, learning, and control of autonomous systems operating in dynamic environments, with an emphasis on addressing the challenges faced on various timescales. For example, autonomous robotic agents need to plan/execute safe paths and avoid imminent collisions given noisy sensory information (short timescale), learn how to interact with other agents (possibly humans) with intents that are not known (medium timescale), and perform complex cooperative tasks given imperfect models and knowledge of the environment and teammate actions (long timescale). These tasks are often constrained to be done using onboard computation and perception, which can add significant complexity to the system. The talk will highlight several recently developed solutions to these challenges that have been implemented to demonstrate high-speed agile flight of a quadrotor in unknown, cluttered environments, autonomous navigation of a ground vehicle in complex indoor environments alongside pedestrians, and real-time cooperative multiagent planning with an onboard deep learning-based perception system.