Tracking particle motion in inertial flows (especially in obstructed geometries) is a computationally daunting proposition. This is further complicated by that fact that the construction of migration maps for particles (as a function of particle location, flow conditions, and particle size) requires several thousands of simulations tracking individual particles. This calls for the development of an efficient, scalable approach for single particle tracking in fluids. We bring together three distinct elements to accomplish this: (a) a parallel octree based adaptive mesh generation framework, (b) a variational multiscale (VMS) based treatment that enables flow condition agnostic simulations (laminar or turbulent)~\cite{Bazilevs07b}, and (c) a variationally consistent immersed boundary method (IBM) to efficiently track moving particles in a background octree mesh~\cite{Xu:2015ig}. This project builds on our existing codes for adaptive meshing (\dendro) and finite elements (\talyfem). We present our adaptive meshing framework that is tailored for the immersed boundary method and experiments demonstrating the scalability of our code to over 1k compute nodes.