In recent years, the open data movement is gaining momentum in the transportation industry with multiple State's Department of Transportation (DOT) launching their own repository of datasets. The quality of data, ease of usage and availability of metadata varies from source to source. There is an imminent need to assess the quality of open data portals to provide agencies a yardstick to measure their performance and draw inspirations from higher ranking portals. We propose a data portal evaluation rubric (DPER) which can serve this purpose. DPER is designed to capture the essence of the National Open Data Policy. The DPER was used to evaluate 43 data portals at the state and national level which provide transportation datasets. DPER evaluates the quality of the portal, the openness of data, and the relevance of its content to the transportation sector. The portal of the State of New York scores the highest due to its user-friendly interface with interactive visualization tools, relevant data content, detailed data information and useful API references for application developers.