Like The Hub?
Join our community.
Join

Adam Lavecchia, Philip Oreopoulos, and Noah Spencer: Too good to be true? How an education program is transforming the lives of at-risk youth

Commentary
Test image credit

This is a test subtitle

In 2007, a consulting firm published an astonishing report. The report concluded that a nonprofit organization called Pathways to Education was responsible for reducing high school dropout rates by 40 percentage points in Toronto’s Regent Park neighbourhood. Such an enormous improvement would be almost unprecedented in evaluations of other similar programs in North America.  

As academic economists, we are generally skeptical people. When we came across this report, our first instinct was that the results were too good to be true. The methods also seemed dubious to us. The report compared students who participated in the non-profit’s programming to students in the same neighbourhood who did not participate, not accounting for the fact that more motivated students may be more likely to participate in the programming. It also didn’t account for the fact that high school graduation rates were rising across the City of Toronto during the 1990s and 2000s. 

Given these shortcomings, we decided to conduct our own research into the effectiveness of Pathways’ program. We started the project expecting to nullify the consulting firm’s findings. What we found instead was a remarkable success story. 

Before telling that story, we should quickly describe the program. Pathways is a non-profit organization that began operating in 2001 in the Regent Park community in Toronto. Over the years, Pathways expanded to 31 communities across the country. It aims to improve the lives of adolescents living in poverty through a comprehensive set of services. The program is organized through four complementary pillars: tailored student social supports, tutoring, mentoring, and financial support. Youth are guided through the program by support workers who they meet with regularly to discuss academic progress, home life, well-being, and any challenges they may be facing. They also meet regularly with academic tutors and volunteer mentors.  

To evaluate whether Pathways was actually succeeding in altering the life trajectories of young people, we compared the outcomes of adolescents eligible for the program to a comparison group of similar students who were not eligible, before and after Pathways was introduced in Regent Park. 

Using this approach, we found that high school graduation and post-secondary enrollment rates increased dramatically for Pathways-eligible students. We also found that eligibility for Pathways increased their annual earnings by 19 percent, employment rates by 14 percent, and decreased social assistance receipt by more than 30 percent. In a new research paper released last month, we find that eligibility for the Pathways program in Regent Park reduced the likelihood a person was charged with a crime by 32 percent. 

It is not just the youth participating in Pathways’ program who benefit from it. Society as a whole benefits from a more educated and productive workforce, from needing to make fewer social assistance payments, and from less crime. We estimate the increased tax revenue from the improvement to labour market outcomes alone outweigh the direct operating costs of Pathways. Accounting for potential improvements to health and reduced expenditures in the criminal justice system and social assistance would further enhance the social return of the program.  

Prime Minister Stephen Harper, left, sits next a girl associated with Pathways to Education Canada, a charitable organization to reducing poverty and providing post-secondary education as they pose for a photo opportunity in Toronto on Thursday, March 3, 2011. Nathan Denette/The Canadian Press.

We think there are three implications for the Pathways Program. 

First, the program is cost-effective in the sense that it generates a high return. If the effects we are finding for Regent Park are the same for other sites (this is our best guess, but for full certainty, the study would have to be replicated for other sites), then an easy case can be made to expand Pathways to other at-risk communities across Canada. 

Second, while the Pathways program we evaluated “pays for itself” in terms of long-run increased tax revenue, lower crime costs, and lower transfer payments, there is a significant time lag between when the up-front investment in the program needs to be made and when the returns are realized. This time difference is measured in decades, not years. Since there are few civil society organizations with investment horizons that span a generation or more, we believe there is a role for government to play in investing in programs like Pathways that generate large societal benefits.  

Finally, we believe Pathways is better as a nimble grassroots effort rather than as a government-designed and delivered program. As Pathways expands across Canada, it must strike the right balance between delivering a program that has been shown to generate benefits and responding to the local needs of the communities it operates in. Our view is that this flexibility is better achieved through investment in grassroots non-profit organizations like Pathways Canada rather than the creation of new federal programs.  

But we wouldn’t be academics if we didn’t say more research needs to be done to help ensure this good news story is actually going on at all locations, and also investigate whether improvements can be made. While we now understand the positive effects of Pathways at its Regent Park location, more research is needed to evaluate whether the Pathways model has been successful in the 30 other communities it now operates in and how it can be improved or costs reduced.

Furthermore, future research should attempt to understand the relative importance of the different features of the Pathways program in generating these positive effects. In particular, this research should attempt to discern whether one or more features of the program (e.g. tutoring, mentoring, or financial supports) primarily drive its long-run impacts or whether the four pillars of the program interact to generate impacts. 

If researchers can answer these questions well, the success of the program that began in Regent Park could expand to help even more at-risk youth. 

‘Nothing lifts the burden of accountability from the PM’: Ian Brodie on Justin Trudeau’s foreign interference testimony

Commentary

Justin Trudeau provided testimony on Wednesday to the Foreign Interference Commission as part of the national public inquiry into foreign interference in Canada’s elections. Parts of his testimony concerned if and to what extent he had been briefed on matters related to the meddling. The Hub’s editor-at-large Sean Speer exchanged with Ian Brodie, the former chief of staff to Prime Minister Stephen Harper, to get an insider’s perspective on the ins and outs of briefing a prime minister. You can find more of Brodie’s commentary and other writings at his Substack here.

SEAN SPEER: As a general rule, the prime minister seems to have suggested that he doesn’t rely on written briefs. He and his office instead typically use oral briefings including for (but not limited to) intelligence and national security matters. My experience with Prime Minister Harper is that he received approximately ten written briefing notes from PMO staff and the Privy Council Office each day. Is that consistent with your own experience, and if so, what do you think is lost by minimizing the use of written briefs?

IAN BRODIE: It’s true, different prime ministers get their briefings in different ways. Mr. Trump responded to “killer graphics” as his CIA director said. In 2006, civil servants had to get used to Mr. Harper reading a lot of long, detailed written notes. Mr. Chretien liked short notes, to the point. Mr. Harper liked more background material. He got so many notes about the 2007 equalization reform, they filled an entire drawer of a filing cabinet! I really looked forward to days when we only got ten briefing notes from PCO.

Written notes had a big advantage. Mr. Harper initialized every note that he read. We knew the date he received every note and the date he returned it for filing. There was never any doubt about what he’d been briefed and when it had been briefed.

Oral briefings are fine, but you need a notetaker to record what was said. With really critical information, I prefer to have a paper trail. Plus, if the PM reads the preliminary material ahead of time, the briefing is more productive.

SEAN SPEER: With respect to specific intelligence briefing notes, the prime minister said that while he reads them when he can, he instead expects advisers to tell him if there’s something important. How does that approach converge or diverge with your experience as the prime minister’s chief of staff?

IAN BRODIE: Every PM relies on others to decide what he needs to see or hear. The fact Mr. Harper read a lot of written briefings every day meant you could send a lot of material his way. In my day, the clerk of the Privy Council had direct access with written notes and verbal briefings. So did I and several PMO staffers. Cabinet ministers, too. That meant there were fewer points of failure. It was rare for a single person to overlook something and leave the PM out of the loop.

But, nothing lifts the burden of accountability from the PM. He appoints people to make sure he knows what he needs to know. I am sure that several people said “Hey, the PM needs to know that China is working hard to get this guy elected as a Liberal,” or “Hey, the Chinese might be threatening an MP’s family because of a vote he took in the House of Commons.” In the case of Michael Chong, I expect he was briefed about threats to the Chong family. I still don’t understand why he didn’t instruct someone to check on Chong’s family, report back to him, and then let Chong know what was going on.

SEAN SPEER: What was Prime Minister Harper’s normal process for consuming national security and intelligence information? Did he have dedicated time each week? Did it typically come in the form of written briefs or oral briefings?

IAN BRODIE: That changed over time. The process became more formal after I left the government. He got a weekly written briefing from the intelligence assessments section at PCO. His various national security advisors would also brief. I visited the main intelligence agencies to make sure we were getting all the right information from them.

SEAN SPEER: In addition to being a chief of staff, you’ve been involved in leadership races and party nominations. The Liberal Party doesn’t prohibit non-residents from participating in its internal voting. The prime minister said that that’s intentional to “encourage wide participation in nomination races.” What are the risks in your mind to such a party policy?

IAN BRODIE: Governments should let political parties govern themselves as they see fit.

Back in 2006-07, we had informal discussions with the opposition parties about requiring proof of citizenship to vote in a federal election. They were opposed. We couldn’t even get an agreement to require photo ID at the ballot box.

Without a photo ID requirement, it’s hard to enforce a citizenship requirement, even when you set one as my party does.

Remember, in the Conservative Party, we let youth members vote even if they aren’t yet old enough to vote in a general election. Every party has wider rules for internal votes than the Elections Act does for general elections.

However, I personally think allowing non-citizens to vote in a party nomination or leadership election is very risky. They are vulnerable to all sorts of threats, as we have seen at the inquiry. Political parties pick the candidates we get to vote for at election time and pick the leaders of the parties that could become prime minister. To my mind, the qualifications to vote in nomination and leadership races should be the same as the qualifications to vote in a general election. Maybe we should have slightly wider rules for a riding board of directors. But when a party is picking candidates and leaders, that should be a tight process.