Understanding Government Transparency vs. National Security Confidentiality

Last updated 2 weeks ago ago. Our resources are updated regularly but please keep in mind that links, programs, policies, and contact information do change.

Two fundamental principles often clash in American government: transparency and secrecy. Transparency means citizens have the right to know what their government is doing. National security confidentiality means protecting sensitive information that could harm the country if revealed.

This tension shapes daily decisions across Washington. When should agencies release information? When must they keep it secret? How do we balance an informed public against genuine security needs? These questions affect everything from military operations to diplomatic cables to intelligence reports.

The stakes are real. Too much secrecy can hide corruption and prevent accountability. Too much openness can endanger lives and compromise national security. Getting this balance right is essential for a healthy democracy.

Defining the Core Principles

What Government Transparency Means

Government transparency is the principle that citizens have an inherent right to know about their government’s activities. It’s not just about making documents available—it’s about actively fostering an informed public that can participate meaningfully in democracy.

Transparency serves several crucial functions:

Accountability: When government operations happen in the open, corruption and mismanagement become much harder to hide. Public scrutiny forces officials to justify their decisions and use resources wisely.

Public participation: An informed citizenry makes better decisions about candidates and policies. Access to comprehensive information empowers people to engage constructively in the political process rather than relying on speculation or partisan talking points.

Better government: Transparency often improves efficiency and effectiveness. When agencies know their work will be scrutinized, they have stronger incentives to perform well and avoid waste.

National asset: The Obama administration emphasized that information held by the federal government is a national asset that should be available to the public in accessible, usable forms.

Platforms like USAFacts.org compile official data to help citizens understand government finances and activities. Official portals like GovInfo.gov provide access to legislative, executive, and judicial documents.

True transparency requires more than just compliance with information requests. It means proactively engaging citizens and incorporating their knowledge into governance—a cultural shift from reactive compliance to active dialogue.

Understanding National Security Confidentiality

National security confidentiality protects information whose unauthorized disclosure could harm the United States’ defense capabilities, foreign relations, or security interests. The logic is straightforward: some information is too dangerous to share publicly.

But what counts as “dangerous” isn’t always clear. Information gets classified only if its disclosure “reasonably could be expected to cause damage to the national security.” This standard involves judgment calls that can vary between officials, agencies, and administrations.

The subjectivity in this phrase—”reasonably could be expected to cause damage”—creates the central tension. What constitutes damage? How reasonable must the expectation be? These questions don’t have precise answers in law, leaving room for both legitimate protection and overuse of secrecy.

This gray area has historically led to over-classification when officials err on the side of caution or when the system gets misused to hide information that’s embarrassing rather than genuinely sensitive.

The Classification System

The U.S. government uses three levels to classify national security information, based on potential damage from unauthorized disclosure:

Top Secret: Information whose disclosure could cause “exceptionally grave damage” to national security. Only the President, Vice President, agency heads, and specifically designated officials can originally classify information at this level.

Secret: Information whose disclosure could cause “serious damage” to national security.

Confidential: Information whose disclosure could cause “damage” to national security.

This system traces back to Executive Order 10290, issued by President Truman in 1951. It formally established these levels and introduced “Top Secret” as a distinct category.

The current framework operates under Executive Order 13526, issued by President Obama in 2009. This order emphasizes a crucial principle: if there’s significant doubt about whether to classify information, it shouldn’t be classified. If there’s doubt about the level, choose the lower one.

Why Both Openness and Secrecy Matter

The Case for Transparency

Robust government transparency isn’t just a nice idea—it’s essential for healthy democracy. The benefits are concrete and measurable.

Building trust and accountability: When citizens can monitor their officials’ actions, it increases public trust in government institutions. People can see how decisions get made and resources get allocated, which reinforces the government’s legitimacy.

Empowering informed participation: Access to timely, comprehensive information helps citizens make better decisions about candidates and policies. An informed electorate can engage in constructive dialogue and advocacy, influencing government in meaningful ways.

Fighting corruption: Transparency is one of the most effective anti-corruption tools. The UN Convention against Corruption explicitly emphasizes transparency’s role in preventing and combating corruption. When processes are open, corrupt practices become much harder to sustain.

Improving efficiency: Public scrutiny drives improvements in government efficiency. When agency operations and spending are open to review, there’s greater incentive to use resources wisely and optimize decision-making.

Informing decision-makers: Transparency helps government officials themselves. Access to quality data on program performance allows officials to assess whether initiatives are achieving their goals and make evidence-based adjustments.

These benefits show that transparency delivers tangible value. The absence of openness carries real costs: increased corruption risk, reduced public trust, inefficient use of taxpayer money, and less engaged citizens.

The Need for Secrets

While transparency is crucial, certain government information must remain confidential to protect vital national security interests. The rationales for secrecy are specific and compelling:

Protecting intelligence operations: Revealing ongoing intelligence activities, the identities of human sources, or collection methods could compromise current operations, endanger lives, and severely undermine future intelligence gathering. Executive Orders often presume that disclosing foreign government information or confidential sources will damage national security.

Safeguarding military capabilities: Information about military plans, weapons systems capabilities and vulnerabilities, ongoing operations, and defense installations must be protected. Disclosing such details could give adversaries strategic advantages and endanger military personnel.

Enabling effective diplomacy: Diplomacy often requires confidentiality. Sensitive negotiations, internal assessments of foreign leaders, and information shared by other nations must be protected to maintain trust and facilitate candid discussions. Premature disclosure could damage relationships and hinder foreign policy objectives.

Preventing terrorism: In today’s security environment, protecting information about vulnerabilities that terrorists could exploit, ongoing investigations, and protective service capabilities is paramount. Disclosure could directly aid terrorist organizations and compromise prevention efforts.

Protecting citizens and institutions: The overarching goal is protecting the nation’s citizens and democratic institutions. Maintaining confidentiality of certain information is essential to prevent threats that could undermine public safety or compromise the nation’s ability to operate securely internationally.

These rationales show that secrecy needs are tied to preventing concrete, potentially severe harm to individuals, operations, defense capabilities, and international relationships. This creates a legitimate counter-argument to demands for absolute transparency.

The Dangers of Imbalance

When the balance tips too heavily toward secrecy, significant dangers emerge that can undermine democratic governance and even harm national security.

Over-classification: This means classifying information unnecessarily, at too high a level, or for too long. There’s broad consensus among officials, experts, and watchdog organizations that the U.S. government classifies far too much information.

Several factors drive over-classification:

• Officials fear severe penalties for leaking more than any consequences for excessive secrecy
• The sheer volume of classification guidance—over 2,000 security classification guidebooks—can be unworkable
• Vague procedures lead officials to err on the side of caution

The consequences are far-reaching:

• Hinders information sharing with allies
• Obscures wasteful spending and program duplication
• Stifles innovation by preventing private sector collaboration
• Impedes effective congressional oversight
• Can damage national security by preventing agencies from learning from mistakes or communicating effectively

“Secret law”: This describes the government’s increasing reliance on secret legal interpretations, classified court opinions (particularly from the Foreign Intelligence Surveillance Court), and unpublished internal regulations to authorize national security activities.

Secret law fundamentally undermines democratic self-government because the public—and often even Congress—cannot know the legal basis for government actions. This prevents meaningful public debate and accountability for potential abuses.

Examples include secret legal memoranda from the Justice Department’s Office of Legal Counsel interpreting government powers, or classified rulings from surveillance courts authorizing broad programs.

Executive Orders explicitly prohibit classifying information to “conceal violations of law, inefficiency, or administrative error; to prevent embarrassment; to restrain competition; or to prevent or delay the release of information that does not require protection in the interest of national security.”

The persistence of over-classification and secret law criticism, despite these formal prohibitions, suggests a significant gap between stated policy and actual practice.

The Legal Framework

The Freedom of Information Act

The Freedom of Information Act (FOIA), enacted in 1966, stands as a landmark piece of U.S. transparency legislation. It grants any person the statutory right, enforceable in court, to request and obtain records from federal executive branch agencies.

FOIA’s primary purpose is promoting an informed citizenry, which is vital to democratic society functioning. By providing public access to government information, FOIA serves as a crucial accountability mechanism and check against corruption or mismanagement.

How FOIA works: Any person—U.S. citizens, foreign nationals, corporations, or other entities—can request agency records. Agencies must disclose requested records unless the information is protected by one of FOIA’s nine statutory exemptions or three law enforcement record exclusions.

These exemptions are generally discretionary, meaning agencies may withhold information but aren’t always required to unless another law mandates withholding.

Key exemptions relevant to transparency and secrecy:

Exemption (b)(1)—Classified National Security Information: This directly links FOIA to the classification system, protecting information properly classified as Top Secret, Secret, or Confidential under Executive Orders for national defense or foreign policy reasons.

Exemption (b)(3)—Information Exempted by Other Statutes: Covers records that other federal statutes specifically shield from disclosure, provided those statutes either require withholding or establish particular criteria.

Exemption (b)(5)—Inter-agency Memoranda: Often called the “deliberative process privilege,” this protects internal government deliberations, advice, recommendations, and opinions that are part of agency decision-making. It also covers attorney-work product and attorney-client communications. Critics argue this exemption gets overused to shield information that should be public.

Exemption (b)(7)—Law Enforcement Records: Protects records compiled for law enforcement purposes, but only if disclosure could cause specific harms like interfering with investigations, disclosing confidential sources, or endangering lives.

Resources: FOIA.gov serves as the central portal for understanding FOIA rights and making effective requests. It offers FAQs, the full statute text, request guidance, and tools to help identify relevant records or the correct agency for submissions.

The Espionage Act of 1917

The Espionage Act of 1917 was enacted shortly after the United States entered World War I. Its original purpose was protecting national defense information from disclosure to enemies and suppressing activities perceived as undermining the war effort.

The Act criminalized conveying information with intent to interfere with U.S. armed forces or promote enemy success, disseminating false reports to interfere with military operations, causing insubordination or disloyalty in the military, and obstructing military recruitment.

Modern controversy: Throughout its history, the Espionage Act has been controversial, particularly regarding its application beyond traditional espionage cases. The Act’s vague and broad language has allowed its use in prosecuting whistleblowers and journalists who disclosed classified information, even when the intent was exposing wrongdoing rather than harming national security.

Such prosecutions raise serious First Amendment concerns about freedom of speech and press. The Obama administration indicted more individuals under the Espionage Act for unauthorized disclosures than all previous administrations combined.

This evolution—from a wartime tool focused on espionage and sedition to a general instrument for punishing unauthorized disclosures—creates direct conflict with government transparency goals and the press’s accountability role.

Executive Orders on Classification

The U.S. classification system is primarily governed by Executive Orders issued by the President. These directives provide detailed rules and procedures that executive branch agencies must follow.

Key developments:

EO 10290 (Truman, 1951): Marked a significant step in formalizing the classification system. It provided the first public definitions for classification levels and officially added “Top Secret” as the highest category.

EO 12356 (Reagan, 1982): Issued during renewed Cold War tensions, this order prescribed a uniform system for classifying, declassifying, and safeguarding national security information. It included provisions stating that information shouldn’t be classified to conceal violations of law, inefficiency, or administrative error.

EO 13526 (Obama, 2009): The current Executive Order governing the classification system. It emphasizes both informing the American people about government activities and protecting information critical to national security.

Key features include:

• Mandatory training for classifiers
• Specific conditions for original classification
• Detailed declassification procedures
• Automatic declassification of records older than 25 years with permanent historical value
• Direction that if there’s doubt about classifying, don’t classify; if there’s doubt about the level, choose the lower one

This progression reveals an iterative process of managing a complex, contentious system. Later orders incorporate mechanisms specifically designed to curb over-classification, suggesting governmental recognition of systemic issues and ongoing efforts to better serve both national security and democratic transparency.

Whistleblower Protection Laws

Whistleblower protection laws encourage and safeguard employees who report waste, fraud, abuse, illegality, or dangers to public health and safety within government. The rationale is that insiders are often best positioned to identify problems, and their disclosures can lead to corrective actions.

The Whistleblower Protection Act (WPA) of 1989 provides protections for most federal civil service employees against adverse personnel actions for disclosing violations of law, rule, or regulation; gross mismanagement; gross waste of funds; abuse of authority; or substantial danger to public health or safety.

National security gaps: When it comes to national security whistleblowers, protection is far more limited and fraught:

Limited coverage: The WPA specifically excludes employees of key intelligence and national security agencies, including the FBI, CIA, Defense Intelligence Agency, and National Security Agency. Uniformed military personnel and many government contractors also aren’t covered.

Criminal exposure: National security whistleblowers often deal with classified information. Disclosing such information, even to report serious wrongdoing, can expose them to criminal prosecution under the Espionage Act, regardless of intent.

Retaliation risks: Retaliation against national security whistleblowers is reportedly common and severe. This includes security clearance revocation (often career-ending), termination, smear campaigns, and retaliatory investigations.

State secrets privilege: In civil litigation, the government may invoke the “state secrets privilege,” which can lead to entire lawsuit dismissals if courts determine that litigation would risk disclosing information harmful to national security.

This creates a fundamental paradox where individuals closest to potential grave misconduct within the national security apparatus may be least protected if they attempt to report issues. This lack of robust protection can deter disclosure of critical information, potentially allowing serious problems to fester unaddressed.

Historical Turning Points

Early American Secrecy and the World Wars

From the nation’s earliest days, some government secrecy was accepted. The Continental Congress often conducted business behind closed doors to facilitate frank debate and protect sensitive negotiations. The Constitution itself acknowledges that Congress may keep parts of its journal secret.

Major international conflicts served as significant catalysts for expanding and formalizing government secrecy.

World War I: This period saw enactment of the Espionage Act of 1917, a sweeping law aimed at protecting military information and suppressing dissent perceived as harmful to the war effort. This marked a significant legislative step toward codifying national security secrecy.

World War II: WWII demands led to dramatic secrecy expansion. Informal protocols for handling sensitive information became codified, giving rise to familiar classification levels of “Confidential,” “Secret,” and “Top Secret.” President Roosevelt’s Executive Order 8381 criminalized unauthorized photographing or sketching of designated military installations or documents.

The war necessitated tight control over information related to military plans, advanced weapon systems like radar and the atomic bomb, and critical intelligence breakthroughs like breaking enemy codes.

These wartime experiences established important precedents and built governmental secrecy systems. The perception of existential threats during major conflicts tended to override typical transparency demands, leading to greater acceptance of government information control in the name of national survival.

The Cold War’s Expansion of Secrecy

The end of World War II didn’t bring a return to pre-war openness. Instead, the Cold War’s onset—a prolonged period of geopolitical tension primarily between the United States and Soviet Union—led to institutionalization and significant expansion of the wartime secrecy apparatus.

Key developments:

The Atomic Energy Act of 1946: This introduced a unique secrecy concept for atomic weapons and nuclear energy information. It created “Restricted Data,” which was deemed “born secret”—automatically classified from creation and only declassifiable through specific Atomic Energy Commission action. This was an extreme departure from traditional transparency principles.

The National Security Act of 1947: This landmark act restructured the U.S. military and intelligence agencies, creating foundational elements of the modern national security state, including the CIA and National Security Council. These new structures inherently operated with significant secrecy.

Skyrocketing classification: The sustained atmosphere of heightened national security concern justified unprecedented peacetime secrecy. By 1985, U.S. government officials were reportedly making around 15 million classification decisions annually, creating an enormous backlog of secret information.

The Cold War era solidified many secrecy structures and practices that persist today. The long-term, pervasive nature of the perceived Soviet threat led to a deeply entrenched culture of secrecy within large parts of the U.S. government.

The Pentagon Papers: Supreme Court Showdown

A critical juncture in the tension between government secrecy and the public’s right to know occurred in 1971 with the Pentagon Papers case. Daniel Ellsberg, a former military analyst, leaked portions of a top-secret Defense Department study detailing U.S. involvement in Vietnam from 1945 to 1967 to The New York Times and Washington Post.

The documents revealed that successive administrations had misled the public about the extent and nature of American involvement in Vietnam.

When newspapers began publishing articles based on the Pentagon Papers, the Nixon administration sought to halt further publication, arguing it would cause irreparable harm to national security and foreign policy conduct. The government obtained temporary restraining orders against the newspapers.

In the landmark decision New York Times Co. v. United States, the Supreme Court ruled 6-3 in favor of the newspapers. The Court’s opinion affirmed that any system of prior restraint comes bearing a “heavy presumption against its constitutional validity.” The government carries a “heavy burden” of showing justification for such restraint.

The Pentagon Papers ruling was a significant victory for press freedom and government transparency. It greatly increased public awareness about Vietnam War realities and likely contributed to growing anti-war sentiment. The decision powerfully underscored the judiciary’s role in checking executive branch national security claims when they impinge upon First Amendment rights.

However, the impact wasn’t absolute. The lack of a single majority opinion and several justices’ acknowledgment that prior restraint might be permissible in truly extreme circumstances meant the fundamental debate over government secrecy limits wasn’t fully settled.

The Post-9/11 Landscape

The September 11, 2001, terrorist attacks marked another profound turning point, dramatically shifting the balance between transparency and national security toward enhanced security measures and increased government secrecy. The attacks created an environment of acute national vulnerability and public demand for robust governmental action.

The Bush administration developed and implemented counterterrorism policies often “behind closed doors” with limited public or congressional debate, leading to programs and legal interpretations that critics said significantly infringed on civil liberties.

Key developments:

The USA PATRIOT Act (2001): Enacted swiftly after the attacks, this act significantly expanded surveillance powers of law enforcement and intelligence agencies. This included enhanced authority to monitor internet activity, emails, and obtain business records, often with reduced judicial oversight.

New security structures: The Department of Homeland Security was established in 2002, consolidating numerous agencies to coordinate national anti-terrorism efforts. “Fusion centers” aimed to improve information sharing between federal, state, and local authorities, though these raised privacy and civil liberties concerns.

Enhanced watchlisting: The Terrorist Screening Center, administered by the FBI, was established to consolidate terrorism watchlists and facilitate sharing terrorism-related information across government and with law enforcement partners.

The post-9/11 period saw significant priority recalibration, where national security concerns often took precedence over traditional transparency and certain civil liberties commitments. The urgency of preventing further catastrophic attacks led to expanded government powers to collect information and operate in secret.

The Digital Age: WikiLeaks and Edward Snowden

The early 21st century witnessed large-scale digital leaks that fundamentally altered the government secrecy and transparency landscape. WikiLeaks and Edward Snowden became synonymous with this phenomenon, demonstrating the vulnerability of vast digital classified information repositories.

WikiLeaks: Founded by Julian Assange in 2007, WikiLeaks gained international prominence for publishing large troves of classified government and corporate documents. Notable releases included hundreds of thousands of U.S. military field reports from Afghanistan and Iraq wars, plus State Department diplomatic cables.

These disclosures caused significant government embarrassment, revealed sensitive diplomatic exchanges, and provided insights into military operations, including civilian casualties instances. The leaks forced governments to re-evaluate information security protocols and led to clampdowns on internal classified network access.

Edward Snowden: In 2013, Edward Snowden, a former NSA contractor, leaked vast classified documents to journalists, revealing the global scale of NSA surveillance programs. His disclosures detailed programs involving bulk collection of Americans’ telephone metadata and extensive monitoring of internet communications worldwide.

Snowden’s revelations triggered widespread public outcry regarding government surveillance extent and its implications for privacy and civil liberties. The disclosures led to significant legal and legislative debates and were a direct catalyst for the USA FREEDOM Act of 2015, which curtailed some NSA bulk data collection practices.

The era of large-scale digital leaks has created a complex, reactive cycle. While such disclosures can force uncomfortable truths into the public domain and sometimes catalyze reforms, they also provoke governments to tighten information controls, enhance cybersecurity measures, and aggressively pursue leakers using laws like the Espionage Act.

Key Players in the Balance

The Executive Branch

The Executive Branch, headed by the President, is the primary originator, holder, and controller of national security information.

Classification authority: The President and designated officials within executive agencies have authority to classify information according to levels and procedures established by Executive Orders. The President holds ultimate authority to classify and declassify information.

Policy implementation: Executive agencies implement rules for safeguarding classified information and respond to public information requests under FOIA.

Executive privilege: The President can assert “executive privilege” to withhold certain internal communications from Congress or courts, particularly those involving presidential advice. This power often creates tension with congressional oversight and public transparency efforts.

Setting the tone: A presidential administration’s leadership and policies heavily influence the overall climate for transparency versus secrecy. Decisions regarding FOIA office resources, classification policy directives, and general public disclosure approaches significantly impact government information accessibility.

The Legislative Branch

Congress plays a crucial constitutional role in shaping the legal framework for transparency and secrecy and overseeing Executive Branch actions.

Lawmaking: Congress enacts laws governing public access to information, such as FOIA, and laws pertaining to national security information protection, such as the Espionage Act.

Oversight: A primary congressional function is Executive Branch oversight to ensure laws are faithfully executed and public funds are appropriately spent. This occurs through committee hearings, investigations, agency reporting requirements, and the appropriations process.

Intelligence oversight: The House Permanent Select Committee on Intelligence and Senate Select Committee on Intelligence have specific responsibilities to oversee U.S. intelligence community activities. These committees are regularly briefed on sensitive intelligence matters and have procedures that could allow classified information disclosure if they determine public interest outweighs potential risks.

While Congress possesses significant formal powers to legislate for transparency and check executive secrecy, its practical effectiveness can be influenced by political dynamics, bipartisan cooperation levels, available resources, and its ability to compel information from an often-resistant Executive Branch.

The Judicial Branch

The federal courts serve as critical arbiters in disputes concerning government transparency and secrecy.

Interpreting laws: Federal courts interpret the meaning and scope of laws like FOIA and the Espionage Act. They rule on lawsuits brought by individuals or organizations seeking access to denied government records and preside over prosecutions involving unauthorized classified information disclosure.

Constitutional adjudication: The judiciary ensures governmental actions and laws conform to the Constitution, including First Amendment protections for freedom of speech and press, and due process rights.

Shaping precedents: Landmark Supreme Court cases, such as New York Times Co. v. United States (the Pentagon Papers case), have profoundly shaped the legal landscape regarding the balance between press freedom and governmental national security claims.

Through their interpretive and adjudicative functions, courts play vital roles in defining government secrecy boundaries and enforcing the public’s right to access information.

Inspectors General

Offices of Inspectors General are independent entities within most federal agencies, established by the Inspector General Act of 1978.

Mission: IGs promote economy, efficiency, and effectiveness in agency operations while preventing and detecting fraud, waste, abuse, and mismanagement in federal programs.

Independence and authority: IGs operate with independence from the agency heads they oversee. They have authority to conduct audits and investigations relating to their agency’s programs and operations. Agency heads generally cannot prohibit or unduly limit IG audits, investigations, or subpoena issuance.

Reporting: IGs report findings and recommendations to both their agency head and Congress.

IGs serve as crucial internal mechanisms for fostering transparency and accountability within government. Their ability to independently investigate and audit agency activities is vital for uncovering problems that might otherwise remain hidden from public view or congressional oversight.

The Media

A free and independent press plays an indispensable role in democratic society by informing the public and acting as a government watchdog.

Investigative journalism: The media, through investigative journalism, often uncover and report on government actions, policies, and potential misconduct that might otherwise remain secret. This function is crucial for holding officials accountable.

Utilizing transparency tools: Journalists frequently rely on tools like FOIA, as well as information from whistleblowers and leaked documents, to gather information not readily available from official channels.

Public discourse platform: The media provide platforms for public discussion and debate about government policies and actions, enabling diverse viewpoints to be heard and contributing to a more informed electorate.

Challenges: The media’s watchdog ability faces several challenges, including increasing media ownership concentration, misinformation proliferation, political and economic pressures that may influence reporting, and significant resource constraints that limit capacity for in-depth investigative journalism.

Civil Society Organizations

Non-governmental organizations are key actors in advocating for government transparency, accountability, and protection of human rights and democratic principles.

Advocacy and research: Organizations like the Brennan Center for Justice, National Security Archive, and Project On Government Oversight (POGO) conduct in-depth research on government practices, analyze secrecy impacts, and advocate for policy reforms to enhance transparency.

Strategic FOIA use and litigation: Many NGOs are sophisticated, persistent FOIA users, filing numerous requests to obtain government documents. They also engage in litigation when information access is denied, often setting important legal precedents.

Public education and mobilization: These organizations educate the public about government secrecy and transparency issues and mobilize public support for reforms.

Civil society organizations often act as specialized, tenacious watchdogs with the expertise, resources, and long-term commitment necessary to challenge government secrecy in ways that individual citizens or resource-limited media outlets may not be able to sustain.

Contemporary Challenges

FOIA’s Persistent Problems

Despite being a cornerstone of U.S. transparency law for over half a century, FOIA faces significant challenges that often undermine its effectiveness.

Backlogs and delays: The most criticized aspect of FOIA administration is pervasive backlogs and delays. Federal agencies are often overwhelmed by request volumes, leading to response times stretching months or years, far exceeding the 20 working-day statutory deadline. As of the third quarter of 2024, the Biden administration reportedly had over 222,000 pending requests.

Excessive redactions: Requesters frequently encounter heavily redacted documents or outright denials based on agencies’ broad application of FOIA exemptions. Exemption 5, protecting inter-agency memoranda, is particularly criticized for alleged overuse. Data from fiscal year 2023 showed over two-thirds of FOIA requests were either redacted, withheld entirely, or denied because no relevant records were found.

Resource constraints: Many agency FOIA offices are chronically underfunded and understaffed. This problem is compounded by ever-increasing volume and complexity of government records, particularly electronic data stored in diverse formats across multiple systems. FOIA compliance cost across the federal government is estimated at over $500 million annually.

Litigation: Frustrated requesters increasingly turn to litigation to enforce FOIA rights. While this provides an appeal mechanism, it further strains limited agency resources and can lead to lengthy court battles.

Reform efforts: Ongoing debates about FOIA reform continue. The FOIA Improvement Act of 2016 introduced several changes, including codifying a “presumption of openness” and requiring agencies to release information unless they reasonably foresee that disclosure would harm an interest protected by an exemption.

Proposed reforms often include suggestions to limit the duration for which records can be withheld under Exemption 5 and efforts to improve clarity and helpfulness of agency determination letters to requesters.

Technology’s Evolving Impact

Technological advancements are profoundly reshaping government transparency and secrecy dynamics, presenting both new opportunities for openness and novel challenges to accountability.

Artificial Intelligence:

AI holds considerable promise for enhancing government efficiency. Agencies are exploring AI for processing vast datasets quickly, detecting fraud and waste, improving service delivery, and assisting in document drafting or information summarization. If AI can streamline routine tasks, it could free resources for more substantive work, potentially including FOIA processing.

However, AI use in government decision-making carries significant risks if not implemented transparently. If AI systems make errors or operate based on biased data, individuals may have no way to understand or challenge automated decisions if underlying algorithms and data are opaque.

To address these concerns, Executive Order 13960 and the Advancing American AI Act mandated creation of federal agency AI use case inventories. These require agencies to publicly document their AI systems, including purpose, data used, potential impacts on rights or safety, and risk mitigation efforts.

Encryption:

Encryption technologies are vital for protecting sensitive government information, securing classified documents, safeguarding communications against interception, and preventing cyber espionage.

However, increasing use of encrypted messaging applications by government officials for official communications presents significant challenges to transparency and open records laws. These apps often feature end-to-end encryption and automatic message deletion. Because these communications typically aren’t stored on government servers, they may not be captured by agency record-keeping systems and often aren’t discoverable in FOIA responses.

Data management:

The volume, variety, and velocity of digital data generated by government agencies create immense challenges for both transparency and effective secrecy. The “explosion in the volume of records and data” makes searching, reviewing, and redacting responsive documents for FOIA requests increasingly complex and time-consuming.

Efforts to centralize and make government data more accessible through portals like USAspending.gov are crucial for transparency. However, persistent issues with data consistency, accuracy, completeness, and usability can limit platform effectiveness.

Citizen Access to Information

Empowering citizens with accessible, understandable information about government activities is central to transparency initiatives. Several resources aim to make public data more readily available:

USAFacts.org: This non-profit, non-partisan organization makes government data more accessible and understandable. USAFacts collects, contextualizes, and publishes official U.S. government data on topics including government finances, population demographics, and policy areas like crime, health, education, and immigration.

GovInfo.gov: Operated by the Government Publishing Office, GovInfo.gov serves as a centralized, free public resource for official publications from all three government branches. This includes congressional bills and resolutions, public and private laws, federal regulations, congressional records and reports, presidential documents, and court opinions.

USAspending.gov: This official website provides transparency regarding federal spending by integrating data from numerous agency financial management systems. The Digital Accountability and Transparency Act (DATA Act) of 2014 sought to improve data quality, consistency, and completeness by establishing government-wide data standards.

FOIA.gov: The federal government’s primary portal for FOIA information, offering guidance for requesters, search tools to help identify correct agencies or existing public records, and functionalities to help create and submit requests.

Our articles make government information more accessible. Please consult a qualified professional for financial, legal, or health advice specific to your circumstances.