DC's Digital Archives Are Drowning in Duplicate Images — And Other Cities Are Pulling Ahead

Washington DC's Office of the Chief Technology Officer is sitting on a backlog of duplicate digital images embedded in public-facing municipal databases — property records, permit filings, historic preservation archives — and the city has yet to deploy any systematic deduplication protocol at scale. Meanwhile, comparable capital cities have been quietly solving this problem for years.

The issue matters now for a specific reason: federal workforce restructuring under the current administration has accelerated the offloading of certain data management responsibilities from federal agencies onto District government systems. When staff reductions ripple through agencies that historically maintained their own digital asset libraries, those files don't disappear — they migrate, often chaotically, into city-adjacent repositories. The DC Office of Planning and the Historic Preservation Office, both headquartered near 1100 4th Street SW, have each absorbed digital collections in the past 18 months that contain significant proportions of redundant image files, according to public budget documents reviewed by The Daily Washington DC.

The problem is visible on the ground. The DC Geographic Information System program, managed through the OCTO, hosts aerial photography and street-level imagery going back to 1999. Archivists and urban planners working with the DC Preservation League — based on 7th Street NW in Penn Quarter — have noted that routine records requests increasingly return duplicate or near-duplicate images that slow processing and inflate storage costs. The Anacostia Community Museum, part of the Smithsonian complex on Fort Place SE, faces a parallel challenge managing its growing digitized collection, portions of which overlap with records held by separate District agencies.

What Other Cities Are Doing

Amsterdam has operated a city-wide digital asset management system since 2021 that runs automated perceptual hashing — a technique that identifies visually similar images even when file names differ — across all municipal photography. The city's Digital City program reported reducing redundant image storage by roughly 40 percent within the first year of full deployment, freeing up server capacity that was being paid for at commercial cloud rates. Seoul's Smart City division implemented a comparable system in 2022 tied to its urban renewal documentation process, flagging duplicate construction-site photography before it enters the permanent record.

London's Southwark Borough Council, which manages a housing stock not unlike DC's mix of historic rowhouses and newer subsidized units, rolled out deduplication tools across its planning portal in late 2023. The council cited a measurable reduction in staff hours spent manually sorting image attachments in planning applications — a task that had been consuming roughly six hours per week per caseworker before automation.

DC has no equivalent program in place. The District's Technology Plan, last updated and published in fiscal year 2025, does reference cloud storage optimization as a priority area, but does not specify image deduplication as a distinct workstream. Storage costs for the OCTO's managed services contracts have not been broken out in line-item detail in publicly available budget documents.

What the District Could Do Next

There are practical options already being piloted closer to home. The New York City Department of City Planning began integrating open-source deduplication libraries into its ZoLa land-use application portal in 2024, a project that required under $200,000 in initial development costs according to budget documents the agency published. Philadelphia's Office of Innovation and Technology announced a similar initiative in early 2025, timed in part to coincide with its digital infrastructure review ahead of World Cup preparations.

For DC, the near-term pressure point is the NoMa and Anacostia corridors, where active development and historic documentation efforts are both generating large volumes of photographic records simultaneously. The DC Zoning Update process, ongoing as of this writing, is adding thousands of image attachments to the public record monthly.

City technologists and archivists who work with these systems would benefit from a formal audit of duplicate image volume across OCTO, the Office of Planning, and the Historic Preservation Office before the fiscal year 2027 budget cycle opens. Without knowing the scale of the problem, any procurement process for a fix remains premature — and storage bills keep accumulating while comparable cities have already closed the gap.