Algorithms: Opinions Embedded in Math … and Ed-tech

A much more accurate definition of an algorithm is that it’s an opinion embedded in math.

So, we do that every time we build algorithms — we curate our data, we define success, we embed our own values into algorithms.

So when people tell you algorithms make thing objective, you say “no, algorithms make things work for the builders of the algorithms.”

In general, we have a situation where algorithms are extremely powerful in our daily lives but there is a barrier between us and the people building them, and those people are typically coming from a kind of homogenous group of people who have their particular incentives — if it’s in a corporate setting, usually profit and not usually a question of fairness for the people who are subject to their algorithms.

So we always have to penetrate this fortress. We have to be able to question the algorithms themselves.

We live in the age of the algorithm – mathematical models are sorting our job applications, curating our online worlds, influencing our elections, and even deciding whether or not we should go to prison. But how much do we really know about them? Former Wall St quant, Cathy O’Neil, exposes the reality behind the AI, and explains how algorithms are just as prone to bias and discrimination as the humans who program them.

Source: The Truth About Algorithms | Cathy O’Neil – YouTube

Follow the video up with Cathy O’Neil’s book “Weapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy”. Here are some selected quotes from the introduction.

And then I made a big change. I quit my job and went to work as a quant for D. E. Shaw, a leading hedge fund. In leaving academia for finance, I carried mathematics from abstract theory into practice. The operations we performed on numbers translated into trillions of dollars sloshing from one account to another. At first I was excited and amazed by working in this new laboratory, the global economy. But in the autumn of 2008, after I’d been there for a bit more than a year, it came crashing down.

The crash made it all too clear that mathematics, once my refuge, was not only deeply entangled in the world’s problems but also fueling many of them. The housing crisis, the collapse of major financial institutions, the rise of unemployment- all had been aided and abetted by mathematicians wielding magic formulas. What’s more, thanks to the extraordinary powers that I loved so much, math was able to combine with technology to multiply the chaos and misfortune, adding efficiency and scale to systems that I now recognized as flawed.

If we had been clear-headed, we all would have taken a step back at this point to figure out how math had been misused and how we could prevent a similar catastrophe in the future. But instead, in the wake of the crisis, new mathematical techniques were hotter than ever, and expanding into still more domains. They churned 24/ 7 through petabytes of information, much of it scraped from social media or e-commerce websites. And increasingly they focused not on the movements of global financial markets but on human beings, on us. Mathematicians and statisticians were studying our desires, movements, and spending power. They were predicting our trustworthiness and calculating our potential as students, workers, lovers, criminals.

This was the Big Data economy, and it promised spectacular gains. A computer program could speed through thousands of résumés or loan applications in a second or two and sort them into neat lists, with the most promising candidates on top. This not only saved time but also was marketed as fair and objective.

Yet I saw trouble. The math-powered applications powering the data economy were based on choices made by fallible human beings. Some of these choices were no doubt made with the best intentions. Nevertheless, many of these models encoded human prejudice, misunderstanding, and bias into the software systems that increasingly managed our lives. Like gods, these mathematical models were opaque, their workings invisible to all but the highest priests in their domain: mathematicians and computer scientists. Their verdicts, even when wrong or harmful, were beyond dispute or appeal. And they tended to punish the poor and the oppressed in our society, while making the rich richer.

I came up with a name for these harmful kinds of models: Weapons of Math Destruction, or WMDs for short.

Equally important, statistical systems require feedback- something to tell them when they’re off track. Statisticians use errors to train their models and make them smarter. If Amazon. ​ com, through a faulty correlation, started recommending lawn care books to teenage girls, the clicks would plummet, and the algorithm would be tweaked until it got it right. Without feedback, however, a statistical engine can continue spinning out faulty and damaging analysis while never learning from its mistakes.

Many of the WMDs I’ll be discussing in this book, including the Washington school district’s value-added model, behave like that. They define their own reality and use it to justify their results. This type of model is self-perpetuating, highly destructive- and very common.

In WMDs, many poisonous assumptions are camouflaged by math and go largely untested and unquestioned.

This underscores another common feature of WMDs. They tend to punish the poor. This is, in part, because they are engineered to evaluate large numbers of people. They specialize in bulk, and they’re cheap. That’s part of their appeal. The wealthy, by contrast, often benefit from personal input. A white-shoe law firm or an exclusive prep school will lean far more on recommendations and face-to-face interviews than will a fast-food chain or a cash-strapped urban school district. The privileged, we’ll see time and again, are processed more by people, the masses by machines.

Needless to say, racists don’t spend a lot of time hunting down reliable data to train their twisted models. And once their model morphs into a belief, it becomes hardwired. It generates poisonous assumptions, yet rarely tests them, settling instead for data that seems to confirm and fortify them. Consequently, racism is the most slovenly of predictive models. It is powered by haphazard data gathering and spurious correlations, reinforced by institutional inequities, and polluted by confirmation bias. In this way, oddly enough, racism operates like many of the WMDs I’ll be describing in this book.

Source: Weapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy (pp. 2-3, pp. 6-8). Crown/Archetype. Kindle Edition.

For how this fits into education, read Weapons of Math Destruction along with Lower Ed and Paying the Price.

Lower Ed shows exploitation of vulnerable. Paying the Price tells how we fail them. Weapons of Math Destruction outlines tools we made to do it.

Source: Kyle Johnson on Twitter

Indeed. These three great books provide a systems view of higher education and its intersections with tech and algorithms. Below, I excerpt from their introductions and book blurbs, provide chapter lists, and select a handful of tweets from authors Tressie McMillan Cottom, Sara Goldrick-Rab, and Cathy O’Neil. They are all active on Twitter and well worth a follow.

Source: Lower Ed, Paying the Price, and Weapons of Math Destruction – Ryan Boren

See also Safiya Umoja Noble’sAlgorithms of Oppression: How Search Engines Reinforce Racism”.

This book is about the power of algorithms in the age of neoliberalism and the ways those digital decisions reinforce oppressive social relationships and enact new modes of racial profiling, which I have termed technological redlining. By making visible the ways that capital, race, and gender are factors in creating unequal conditions, I am bringing light to various forms of technological redlining that are on the rise. The near-ubiquitous use of algorithmically driven software, both visible and invisible to everyday people, demands a closer inspection of what values are prioritized in such automated decision-making systems. Typically, the practice of redlining has been most often used in real estate and banking circles, creating and deepening inequalities by race, such that, for example, people of color are more likely to pay higher interest rates or premiums just because they are Black or Latino, especially if they live in low-income neighborhoods. On the Internet and in our everyday uses of technology, discrimination is also embedded in computer code and, increasingly, in artificial intelligence technologies that we are reliant on, by choice or not. I believe that artificial intelligence will become a major human rights issue in the twenty-first century. We are only beginning to understand the long-term consequences of these decision-making tools in both masking and deepening social inequality. This book is just the start of trying to make these consequences visible. There will be many more, by myself and others, who will try to make sense of the consequences of automated decision making through algorithms in society.

Part of the challenge of understanding algorithmic oppression is to understand that mathematical formulations to drive automated decisions are made by human beings. While we often think of terms such as “big data” and “algorithms” as being benign, neutral, or objective, they are anything but. The people who make these decisions hold all types of values, many of which openly promote racism, sexism, and false notions of meritocracy, which is well documented in studies of Silicon Valley and other tech corridors.

Source: Algorithms of Oppression: How Search Engines Reinforce Racism (Kindle Locations 162-177). NYU Press. Kindle Edition.

It was about slavery.

Do the racists in your family claim the Civil War wasn’t about slavery but instead about states’ rights?

Yeah, mine too. A steady diet of the Southern Strategy and the Confederate Catechism is evident in their morally convenient ahistorical revisionism.

Civil War lessons often depend on where the classroom is.” In the classroom of our family, we teach our kids to consult primary sources, in this case the declarations of secession of South Carolina, Georgia, Mississippi, and our home state of Texas as well as the ordinances of secession.

Historian Kevin Kruse offered a helpful breakdown of these declarations on Twitter.

Here’s an embed of the video mentioned in the thread.

If that institution was discredited in the eyes of the world, then the Confederacy itself would be discredited in the eyes of history. So, it became a psychological necessity, I think, to deny that the Civil War was about slavery.

Source: Was the Civil War about Slavery? When did States Rights Become A Popular Explanation for Secession? – YouTube

Historian Kevin Gannon offered a great thread on the history of Robert E. Lee.

What about non-slaveholding southerners fighting in the Civil War? Historian Keri Leigh Merritt explains how oligarchy, then as now, brutalized and divided us all.

One of my favorite resources for teaching American history and social studies is “Lies My Teacher Told Me: Everything Your American History Textbook Got Wrong”. Here it is on the states’ rights argument and how textbooks have misrepresented the Civil War.

Slavery was the underlying reason that South Carolina, followed by ten other states, left the Union. In 1860, leaders of the state were perfectly clear about why they were seceding. On Christmas Eve, they signed a “Declaration of the Immediate Causes Which Induce and Justify the Secession of South Carolina from the Federal Union.” Their first grievance was “that fourteen of the States have deliberately refused, for years past, to fulfill their constitutional obligations,” specifically this clause, which they quote: “No person held to service or labour in one State, under the laws thereof, escaping into another, shall, in consequence of any law or regulation therein, be discharged from such service or labour, but shall be delivered up . . .” This is of course the Fugitive Slave Clause, under whose authority Congress had passed the Fugitive Slave Act of 1850, which South Carolina of course approved. This measure required officers of the law and even private citizens in free states to participate in capturing and returning African Americans when whites claimed them to be their slaves. This made the free states complicit with slavery. They wriggled around, trying to avoid full compliance. Pennsylvania, for example, passed a law recognizing the supremacy of the federal act but pointing out that Pennsylvanians still had the right to determine pay for their officers of the law, and they refused to pay for time spent capturing and returning alleged slaves. South Carolina attacked such displays of states’ rights:

But an increasing hostility on the part of the non-slaveholding States to the institution of slavery, has led to a disregard of their obligations. . . . The States of Maine, New Hampshire, Vermont, Massachusetts, Connecticut, Rhode Island, New York, Pennsylvania, Illinois, Indiana, Michigan, Wisconsin and Iowa, have enacted laws which either nullify the Acts of Congress or render useless any attempt to execute them.

Thus South Carolina opposed states’ rights when claimed by free states. This is understandable. Historically, whatever faction has been out of power in America has pushed for states’ rights. White Southerners dominated the executive and judicial branches of the federal government throughout the 1850s—and through the Democratic Party, the legislative branch as well—so of course they opposed states’ rights. Slave owners were delighted when Supreme Court Chief Justice Taney decided in 1857 that throughout the nation, irrespective of the wishes of state or territorial governments, blacks had no rights that whites must respect. Slave owners pushed President Buchanan to use federal power to legitimize slaveholding in Kansas the next year. Only after they lost control of the executive branch in the 1860 election did slave owners begin to suggest limiting federal power.

South Carolina’s leaders went on to condemn New York for denying “even the right of transit for a slave” and other Northern states for letting African Americans vote. Before the Civil War, these matters were states’ rights. Nevertheless, South Carolina claimed the right to determine whether New York could prohibit slavery within New York or Vermont could define citizenship in Vermont. Carolinians also contested the rights of residents of other states even to think differently about their peculiar institution, giving as another reason for secession that Northerners “have denounced as sinful the institution of slavery.” In short, slavery permeates the document from start to finish. Of course, the election of Lincoln provided the trigger, but the abiding purpose of secession was to protect, maintain, and enhance slavery. Nor was South Carolina unusual; other states used similar language when they seceded.

Despite this clear evidence, before 1970 many textbooks held that almost anything but slavery—differences over tariffs and internal improvements, the conflict between agrarian South and industrial North, and especially “states’ rights”—led to secession. This was a form of Southern apologetics. Never was there any excuse for such bad scholarship, and in the aftermath of the civil rights movement most textbook authors came to agree with Abraham Lincoln in his Second Inaugural “that slavery was somehow the cause of the war.” As The United States—A History of the Republic put it in 1981, “At the center of the conflict was slavery, the issue that would not go away.”

Source: Loewen, James W.. Lies My Teacher Told Me: Everything Your American History Textbook Got Wrong (pp. 139-140). The New Press. Kindle Edition.

This is hard history, but as the states’ rights racists in our families reveal, we have to teach it and do a much better job than we have done so far.

It is often said that slavery was our country’s original sin, but it is much more than that. Slavery is our country’s origin. It was responsible for the growth of the American colonies, transforming them from far-flung, forgotten outposts of the British Empire to glimmering jewels in the crown of England. And slavery was a driving power behind the new nation’s territorial expansion and industrial maturation, making the United States a powerful force in the Americas and beyond.

Slavery was also our country’s Achilles heel, responsible for its near undoing. When the southern states seceded, they did so expressly to preserve slavery. So wholly dependent were white Southerners on the institution that they took up arms against their own to keep African Americans in bondage. They simply could not allow a world in which they did not have absolute authority to control black labor—and to regulate black behavior.

The central role that slavery played in the development of the United States is beyond dispute. And yet, we the people do not like to talk about slavery, or even think about it, much less teach it or learn it. The implications of doing so unnerve us. If the cornerstone of the Confederacy was slavery, then what does that say about those who revere the people who took up arms to keep African Americans in chains? If James Madison, the principal architect of the Constitution, could hold people in bondage his entire life, refusing to free a single soul even upon his death, then what does that say about our nation’s founders? About our nation itself?

Slavery is hard history. It is hard to comprehend the inhumanity that defined it. It is hard to discuss the violence that sustained it. It is hard to teach the ideology of white supremacy that justified it. And it is hard to learn about those who abided it.

We the people have a deep-seated aversion to hard history because we are uncomfortable with the implications it raises about the past as well as the present.

We the people would much rather have the Disney version of history, in which villains are easily spotted, suffering never lasts long, heroes invariably prevail and life always gets better. We prefer to pick and choose what aspects of the past to hold on to, gladly jettisoning that which makes us uneasy. We enjoy thinking about Thomas Jefferson proclaiming, “All men are created equal.” But we are deeply troubled by the prospect of the enslaved woman Sally Hemings, who bore him six children, declaring, “Me too.”

Literary performer and educator Regie Gibson had the truth of it when he said, “Our problem as Americans is we actually hate history. What we love is nostalgia.”

Slavery isn’t in the past. It’s in the headlines.

These recent events reveal, at least in part, how American schools are failing to teach a critical and essential portion of the nation’s legacy-the history and continuing impact of chattel slavery. Research for this report reveals that high school students don’t know much about the history of slavery in the United States, with only 8 percent able to identify it as the central cause of the Civil War. This should not be surprising, given that most adults wrongly identify “states’ rights” as the cause. Widespread ignorance about slavery, the antebellum South and the Confederacy persists to the present day, and is on display in controversies over monument removal in places like New Orleans, Louisiana, and Charlottesville, Virginia, where protests turned deadly in the summer of 2017. Students and adults alike may even hold fringe beliefs, including notions propagated by white nationalists, such as the idea that slavery wasn’t “so bad,” or that the Irish were enslaved.9 Few Americans acknowledge the role slavery played in states outside the South.

Teachers struggle to do justice to the nation’s legacy of racial injustice. They are poorly served by state standards and frameworks, popular textbooks and even their own academic preparation. For this report, we surveyed more than 1,700 social studies teachers across the country. A bare majority say they feel competent to teach about slavery. Most say that the available resources and preparation programs have failed them. Almost all regret this deficiency, recognizing that teaching the history of slavery is essential. When we reviewed a set of popular history textbooks, we saw why teachers felt a lack of support: Texts fail in key areas, including connecting slavery to the present and portraying the diversity of the experiences of the enslaved. State content standards, which are meant to set clear expectations for instruction, are scattershot at best, often making puzzling choices such as teaching about Harriet Tubman long before slavery, or equivocating on the cause of the Civil War. When we consider the available landscape of materials and expectations, it is no wonder that teachers struggle.

Source: Teaching Hard History: American Slavery

Mass Graves, Peonage, and Convict Leasing

CW: slavery, violent racism, mass graves

A mass grave was unearthed in Sugar Land, Texas, where imprisoned and enslaved people worked sugar plantations during the slavery and Jim Crow eras. “One local historian believes this is the tip of the iceberg — that similar sites span the surrounding area.”

Those prisoners were people trapped in racist peonage.

But the most corrupt and abusive peonage occurred in concert with southern state and county government. In the south, many black men were picked up for minor crimes or on trumped-up charges, and, when faced with staggering fines and court fees, forced to work for a local employer would who pay their fines for them. Southern states also leased their convicts en mass to local industrialists. The paperwork and debt record of individual prisoners was often lost, and these men found themselves trapped in inescapable situations.

Source: Slavery v. Peonage | Slavery By Another Name Bento | PBS

The plantation soon became notorious across the state as the “Hellhole on the Brazos.”

Source: Confronting Sugar Land’s Forgotten History

Let’s get some context for these mass graves from Michelle Alexander’s “The New Jim Crow: Mass Incarceration in the Age of Colorblindness”. Convict leasing was racist, brutal, and unjust.

Black people found themselves yet again powerless and relegated to convict leasing camps that were, in many ways, worse than slavery. Sunshine gave way to darkness, and the Jim Crow system of segregation emerged—a system that put black people nearly back where they began, in a subordinate racial caste.

Once again, vagrancy laws and other laws defining activities such as “mischief” and “insulting gestures” as crimes were enforced vigorously against blacks. The aggressive enforcement of these criminal offenses opened up an enormous market for convict leasing, in which prisoners were contracted out as laborers to the highest private bidder. Douglas Blackmon, in Slavery by Another Name, describes how tens of thousands of African Americans were arbitrarily arrested during this period, many of them hit with court costs and fines, which had to be worked off in order to secure their release. With no means to pay off their “debts,” prisoners were sold as forced laborers to lumber camps, brickyards, railroads, farms, plantations, and dozens of corporations throughout the South. Death rates were shockingly high, for the private contractors had no interest in the health and well-being of their laborers, unlike the earlier slave-owners who needed their slaves, at a minimum, to be healthy enough to survive hard labor. Laborers were subject to almost continual lashing by long horse whips, and those who collapsed due to injuries or exhaustion were often left to die.

Convicts had no meaningful legal rights at this time and no effective redress. They were understood, quite literally, to be slaves of the state. The Thirteenth Amendment to the U.S. Constitution had abolished slavery but allowed one major exception: slavery remained appropriate as punishment for a crime. In a landmark decision by the Virginia Supreme Court, Ruffin v. Commonwealth, issued at the height of Southern Redemption, the court put to rest any notion that convicts were legally distinguishable from slaves:

For a time, during his service in the penitentiary, he is in a state of penal servitude to the State. He has, as a consequence of his crime, not only forfeited his liberty, but all his personal rights except those which the law in its humanity accords to him. He is for the time being a slave of the State. He is civiliter mortus; and his estate, if he has any, is administered like that of a dead man.

The state of Mississippi eventually moved from hiring convict labor to organizing its own convict labor camp, known as Parchman Farm. It was not alone. During the decade following Redemption, the convict population grew ten times faster than the general population: “Prisoners became younger and blacker, and the length of their sentences soared.” It was the nation’s first prison boom and, as they are today, the prisoners were disproportionately black.

This harsh reality harks back to the days after the Civil War, when former slaves and their descendents were arrested for minor violations, slapped with heavy fines, and then imprisoned until they could pay their debts. The only means to pay off their debts was through labor on plantations and farms—known as convict leasing—or in prisons that had been converted to work farms. Paid next to nothing, convicts were effectively enslaved in perpetuity, as they were unable to earn enough to pay off their debts. Today, many inmates work in prison, typically earning far less than the minimum wage—often less than $3 per hour, sometimes as little as 25 cents. Their accounts are then “charged” for various expenses related to their incarceration, making it impossible for them to save the money that otherwise would allow them to pay off their debts or help them make a successful transition when released from prison. Prisoners are typically released with only the clothes on their backs and a pittance in gate money. Sometimes the money is barely enough to cover the cost of a bus ticket back home.

Source: The New Jim Crow – Mass Incarceration in the Age of Colorblindness Michelle Alexander

This Texas Monthly piece looks into Sugar Land’s history, its willful amnesia, and the connections between convict leasing and today’s mass incarceration.

Off U.S. 90, behind a bustling shopping center, is a small cemetery surrounded by two concentric rings of chain-link fence. Inside are several dozen crumbling headstones, inscribed with the names and prison numbers of the convicts who died working the sugar plantations that gave the city its name. Most of the convicts died young.

Harvesting cane was even more arduous than picking cotton. Slaves worked around the clock during harvest season to cut the sugarcane, press out the cane juice, boil it down, and then pack the finished product onto trains to be shipped around the country. “Sugar work was about as bad as you can imagine,” said Sean Kelley, a historian of early American history at the University of Essex. “People got sick, they died. Women’s fertility rates plummeted. Europeans quickly discovered that you couldn’t get people to work in this voluntarily, which is why there’s a strong historical linkage between sugar and slavery.”

Cunningham and Ellis survived the abolition of slavery by finding a new source of cheap labor: the Texas prison system. Although they weren’t the first growers to use convict labor, they were the biggest: in 1878 they signed a contract with the state to lease Texas’s entire prison population. This was perfectly legal, since the Thirteenth Amendment, which outlawed slavery, made one very consequential exception: “Neither slavery nor involuntary servitude, except as a punishment for crime whereof the party shall have been duly convicted, shall exist within the United States, or any place subject to their jurisdiction.” (Italics added.)

After abolition, the prison population exploded, disproportionately with black men. Unable to house and feed all the new prisoners, the state began renting them out to private companies, who were grateful for the supply of cheap labor.

The working conditions in Cunningham and Ellis’s sugar fields were as bad or worse than they had been on the slave plantations. Mosquito-borne epidemics, frequent beatings, and a lack of medical care resulted in a 3 percent annual mortality rate. The plantation soon became notorious across the state as the “Hellhole on the Brazos.”

Almost totally missing from the city’s historical memory, Moore realized, is any trace of the slave and convict labor that made the area’s sugar plantations so profitable. The history on the official city website makes no mention of slavery or convict leasing.

“Convict leasing is the missing link between slavery, which everyone knows about, and the disparate impact of the criminal justice system on African Americans today.”

Source: Confronting Sugar Land’s Forgotten History

See also,

As a Twitter thread:

Why confronting this matters: