The COVID-19 pandemic resulted in profound changes in mobility patterns and altered travel behaviors locally and globally. As a result, movement metrics have widely been used by researchers and policy makers as indicators to study, model, and mitigate the impacts of the COVID-19 pandemic. However, the veracity and variability of these mobility metrics have not been studied. This paper provides a systematic review of mobility and social distancing metrics available to researchers during the pandemic in 2020 in the United States. Twenty-six indices across nine different sources are analyzed and assessed with respect to their spatial and temporal coverage as well as sample representativeness at the county-level. Finally global and local indicators of spatial association are computed to explore spatial and temporal heterogeneity in mobility patterns. The structure of underlying changes in mobility and social distancing is examined in different US counties and across different data sets. We argue that a single measure might not describe all aspects of mobility perfectly.