Management & Operations

Report says NY MTA misled public for years about subway delays

Posted on February 13, 2019

A new report released by the New York City Comptroller reveals that the MTA has misrepresented subway performance and delay data for years and shifted the focus from the subways’ real operational problems. Photo: Patrick Cashin/MTA
A new report released by the New York City Comptroller reveals that the MTA has misrepresented subway performance and delay data for years and shifted the focus from the subways’ real operational problems. Photo: Patrick Cashin/MTA

A new report released by the New York City Comptroller reveals that the MTA has misrepresented subway performance and delay data for years and shifted the focus from the subways’ real operational problems. These practices endured despite repeated warnings from agency insiders that much of the delay data released to the public was all but meaningless.

The MTA responded that the report did not recognize the progress the agency has made since 2017, Crain's New York Business reported. Byford assumed his post in January 2018.

"We appreciate the comptroller's focus on subway performance," an MTA statement said, "but this report is more history and politics than news, focusing on rejected practices of the past while glossing over recent reforms and NYC Transit's aggressive pursuit of additional transparency and accountability."

The investigation, based on internal documents primarily prepared by the MTA’s Performance Analysis Unit, as well as interviews with key MTA officials, makes clear that a history of distorted metrics has prevented transparency into what is going on underground in the subways. More specifically, the investigation found that the MTA failed to provide clear guidance on how to identify the causes of millions of delays, relied on distorted metrics to report system progress, and hid delay data where causes were unknown.

“While the new leadership deserves credit for trying to clean things up, the MTA still has a long way to go to ensure accuracy and reliability and to regain the public’s trust,” said Comptroller Scott M. Stringer. 

The Comptroller’s investigation found:

Incomplete and Unreliable Delay Data: From mid-2015 forward, numerous in-house MTA analyses concluded that the Authority’s internal systems could not accurately identify the causes of delays and, in particular, chronically misattributed delays to “Overcrowding.” A July 2015 internal MTA memo described the methodological breakdowns in stark terms: “No policy or guidance exists on how dispatchers should properly identify the cause of a particular delay or on how delays should be assigned to incidents… Dispatchers rely on train crews to report the cause of delays, and these explanations are suspect.”

Meanwhile, in January 2016, another memo noted that “much of the delay data is incomplete or unreliable, particularly the classification/categorization of delays and the assignment of delays to particular incidents.” Despite multiple warnings, MTA officials continued for years to publicly promote misleading information around delays, resulting in the agency being portrayed in a more positive light.

Meaningless and Disappearing “Wait Assessment” Results: Throughout 2016, MTA officials repeatedly asserted that subway service was improving based on apparent increases in “Wait Assessment” scores, a metric intended to measure the reliability and consistency of train service and touted by the MTA as its most important indicator of customer service quality.

However, internal analyses obtained by the Comptroller’s Office showed MTA analysts warning that alleged improvements in “Wait Assessment” were statistically insignificant and likely the result of “sample error.” After technological advancements in data collection finally made clear that “Wait Assessment” scores had actually gotten worse, not better, the MTA quietly stopped touting the metric without ever acknowledging that the prior reporting had been incorrect, according to the report.

Hidden “Unknown” Delay Causes: For nearly a decade, the MTA further distorted its disclosures by effectively hiding delays it had chalked up internally to “Unknown” causes. Instead of publicly listing these delays simply as “Unknown” in a separate category, MTA officials folded them into other publicly identified delay causes – burying the “Unknowns” without any explanation.

In this way, in Monthly Operations Reports provided to the public and the MTA Board from 2013 through mid-2018, the MTA hid 525,710 delays internally grouped under “Unknown.”

Flaws in New Metrics: Similar to the MTA’s misattribution of “Unknown” delays, the MTA’s current reporting of “Major Incidents” – defined as any event that delays 50 or more trains – conceals critical information. Specifically, “Major Incidents” exclude a large category of service disruptions tracked internally by the MTA – “Planned Work,” which includes track and signal maintenance that regularly bogs down whole subway lines. To date, the MTA has yet to acknowledge publicly that its “Major Incidents” metric excludes such work and, as such, fails to capture what is actually going on underground.

Under New York City Transit President Andy Byford, the MTA has recently acknowledged some of these issues, introduced reforms, and begun to reduce delays. Yet the report reveals systemic deficiencies, some of which remain embedded in the MTA’s performance reporting and continue to obscure the true causes of delays.

Comptroller Stringer is calling on the MTA to:

  • Structure public reporting of delay data to maximize transparency, reliability and accountability.
  • Publicly define all delay categories, specifically indicating what each one includes and, as necessary, omits.
  • Ensure that all protocols relevant to performance reporting are formally codified in official policies and procedures, including establishing written definitions and instructions for all key terms, data categories, and work protocols.
  • Adequately train all relevant personnel on protocols relevant to performance reporting.
  • In the context of public reports of “Major Incidents,” provide the public with information about all categories of service disruptions tracked as incidents within its Subway Incident Reporting System (SIRS) that cause 50 or more delays, including specifically “Planned Work.”
  • Transparently disclose in each Monthly Operations Report and on the MTA’s subway performance dashboard, the methodologies used to calculate performance metrics, including all exceptions and revisions to those methodologies, and including methodological weaknesses.
  • Make available on the MTA’s website or through an Open Data portal each month all data in the SIRS database and any other databases relied on for public reporting.

View comments or post a comment on this story. (1 Comment)

More News

WSP USA names new leader for transit, rail zero-emissions tech

John Drayton will support the company’s vehicle initiatives, providing national strategy development and managerial expertise.

NYCT, Visa set to launch contactless cards to ride subways and buses

Chas also worked on new system to help the MTA accept tap-to-pay for its new OMNY fare payment system.

First Transit names Perez as VP of safety

Perez has more than 23 years of safety experience in areas such as manufacturing, waste management and food processing.

APTA chair talks agenda, how to be a better board member

David M. Stackrow Sr. was elected chair of APTA's board of directors last fall.

TARC Turns to tech to become an integrated mobility solutions provider

Customer amenities continue to set the agency apart and take it closer to its larger goal of becoming the mobility manager for the region.

See More News

Post a Comment

Post Comment

Comments (1)

More From The World's Largest Fleet Publisher

Automotive Fleet

The Car and truck fleet and leasing management magazine

Business Fleet

managing 10-50 company vehicles

Fleet Financials

Executive vehicle management

Government Fleet

managing public sector vehicles & equipment

TruckingInfo.com

THE COMMERCIAL TRUCK INDUSTRY’S MOST IN-DEPTH INFORMATION SOURCE

Work Truck Magazine

The number 1 resource for vocational truck fleets

Schoolbus Fleet

Serving school transportation professionals in the U.S. and Canada

LCT Magazine

Global Resource For Limousine and Bus Transportation