Meet Chicago's Interrupters…

On the stoop of a house on a dilapidated block in Englewood, the south side Chicago neighbourhood that tops the city's statistics for murder, drug addiction, teen pregnancy and most of other indices of social dysfunction, are eight young African-American men and two or three women. It's an oven-hot summer afternoon and the group is kicking back, drinking, shouting and laughing.

"I don't like crowd scenes," says Shango, a member of the city's anti-violence project, CeaseFire, as we pull up outside. He explains that such gatherings increase the chances of becoming a victim of a drive-by shooting.

The street we're in stands in the middle of a few blocks that have seen three murders in recent days, and countless more in the previous months and years. "Can't no anybody park up on this block," says Shango, who beneath his dreads wears an expression of mournful unflappability. His companion, TJ, a former prisoner and one of CeaseFire's more seasoned outreach workers, tells me that summer is the "killing season" because there's no refuge for grievances. In winter the freezing weather forces people inside, where tempers have time to cool.

One of the group on the stoop, a slim-built guy with a tattoo that crawls up out of the top of his vest like a weed, hobbles over to us on the sidewalk. His left foot is in plaster, the result of a high velocity encounter with a bullet.

This is Dee, a reformed veteran of the south side's street wars. With his tattoos, gunshot wound and lively choice of company, he may not be Chicago's answer to Mahatma Gandhi, but he is nonetheless a new recruit to the cause of peace. "My life is on the right path today," he tells me, as Shango suspiciously eyes each car that turns on to the block. "I'm doing all I can for the betterment of my life, to keep me out of trouble. It took me 18 years to get my life into shape and it will take me another 18 years or more to make this transition. That's why I'm asking for the full support of the people for me to do what's necessary to do."

It's a noble little speech, and all the more commendable because the last time I saw Dee he was being attacked with a brick. That was in a scene from a remarkable new documentary film called The Interrupters that chronicles a radical approach to urban violence. The film is made by Steve James, who produced and directed the internationally acclaimed Hoop Dreams, which followed the plight of two African-American teenagers trying to become professional basketball players. That documentary was by turns touching, funny, distressing and uplifting, and if anything The Interrupters concocts a still more potent combination of conflicting emotions.

The tagline of the film is "A year in the life of a city grappling with urban violence". In most American cities, including Chicago, violence has actually been declining since the 1990s, with homicide rates plummeting. In 1990 there were 850 murders in Chicago. Last year there were 435. But that's still about three times as many as London, a city with three times the population of Chicago.

Nor has the decrease in homicide spread equally to all urban areas. As one recent report by Northwestern University's Institute for Policy Research put it: "The long-term reduction that is taking place is not as substantial in African-American communities."

There is a tendency to see the problem, which has survived a series of local and federal initiatives, as intractable, the tragic product of a history of deprivation and discrimination. Whether out of fear, indifference or a sense of impotence, the general population has learned to turn away, like commuters speeding by on the freeways to the suburbs, unseeingly passing over the squalor. The Interrupters is an invitation to take a close-up look at the hostile conditions in which a sizeable minority of Americans continue to live in an era that's produced the nation's first black president. "This," says the civil rights veteran Jesse Jackson early on in the film, "is what a war zone looks like." Had the filmmakers simply ventured into the "war zone", the result might have been a form of anthropological voyeurism, full of mindless bloodshed and paralysed despair. But while the camera, as Dee can confirm, doesn't shrink from scenes of violence, what makes the film a moving testament to humanity is the irrepressible spirit it captures in the work of CeaseFire, whose slogan reads "Stop. Killing. People."

Set up by an epidemiologist named Dr Gary Slutkin, CeaseFire approaches violence from a public health perspective. Slutkin believes that violence should be treated not as a moral crisis but as a preventable disease. His background is in primary health care and among his past achievements is the development of a successful strategy to combat Aids in Uganda. With most epidemics, he points out, it's a change in behaviour that stops the spread of the disease.

Progressive thinking has traditionally taken a holistic view of violence, as a symptom of a much larger complex of social injustices – poverty, unemployment, family breakdown, racism, drug abuse, alienation. Address these underlying causes, the theory goes, and violence withers. Slutkin argues that it makes more sense to target violence itself. "Maybe there's completely defective thinking [in the conventional view]," he says in the film. "Maybe a lot of things will improve when the violence is reduced."

Chicago has long been associated with gangsters, but in the popular imagination mostly of the historical kind. We think of Al Capone, the St Valentine's Day massacre, bootlegging. That was all back in the 1920s and 30s, Capone was sent to prison for tax evasion, and the world moved on. The gangsters, however, didn't go away. It's just that, outside Chicago, no one took much notice of them.

Jeff Fort, for example, is a name that means little to most Americans, let alone Europeans, but he is a legend, and indeed a hero, among Chicago's African-American community. A mixture of Don Corleone and Robin Hood, with a wardrobe from the movie Shaft, Fort was the head of gangs that boasted thousands of foot soldiers in the 60s and 70s. Known as the Black Prince, he presented himself as a community leader and such was his influence that he was once invited to the White House by President Nixon.

But as with Capone, the law eventually caught up with him. In 1987, while already in prison on another charge, he was found guilty of conspiring with Colonel Gaddafi's Libyan regime to perform acts of terrorism in the US. The following year another court found him guilty of commissioning the murder of a rival gang leader. His combined sentence amounted to 155 years.

The federal investigations of leaders like Fort led to the collapse of hierarchical, well-organised African-American gangs in Chicago, and the near simultaneous demolition of the city's notorious housing projects further loosened the control they held over black ghettos and the drug economy. Nowadays, the supergangs have broken up and in their place are what are known as "cliques", small groups of boys and young men, bound together by geographical allegiances that may extend no further than the block on which they reside. Many street disputes are not gang or even clique related, but the climate of violence created by the gangs, with their ready access to arms, means that a Hobbesian, kill-or-be-killed mentality can afflict even the most minor altercations. In that situation a lot of CeaseFire's traditional community activism – civic meetings, leafleting, protest marches – has little traction. So in 2004 Slutkin approved the plan of a one-time street hustler called Tio Hardiman to set up a specialist group within CeaseFire made up largely of ex-cons and former gang members. As Hardiman puts it: "We brought the dirty dozen to the table."

The team was called the violence interrupters, and its job was to mediate street disputes so as to stop the escalation to weapons use, injury and murder. The interrupters don't take sides, they're not police informers, and they don't seek to challenge gangs, drug distribution or other criminal activities. They simply aim to intervene at critical points to enable both sides in a conflict to take a step back. The motivating ethic for all their work is that violence need not be the inevitable outcome of street squabbles.

The first field operation was in one small neighbourhood in north-west Chicago that had suffered 12 homicides in 2003. CeaseFire placed two interrupters on the ground, and in 2004 there were no homicides recorded at all. The violence interrupters are run by Hardiman, who works out of the main CeaseFire office in a building that forms part of the University of Chicago's medical school. On the walls of his office are photographs of Bob Marley, Nelson Mandela, Muhammad Ali and Huey Newton, the infamous Black Panther leader. A light-skinned man with a bald crop, Hardiman is a born showman with an almost musical gift for talking, but he is also a serious student of black history. I mention to him Fox Butterfield's classic examination of African-American violence, All God's Children, which traces the legacy back to the brutality of slavery. Butterfield also contends that the Southern white tradition of defending "honour" with lethal duels transmuted in today's no less fatal black code of "respect".

"I'm glad you brought that up," he says, having read the book himself, "because no one really wants to talk about the colonisers' impact on the reason why Caribbean African people are violent." He runs through a potted history from the Middle Passage, taking in sharecropping, segregation, lynchings, the Ku Klux Klan, and the murder of Martin Luther King. "You can't make excuses for the violence," he says, "but we were taught violence."

He grew up in "an environment where violence was the norm. It was like eating breakfast every morning". The mistake many outsiders make, he says, is they assume that because something is normalised, it becomes less threatening. "People have to understand the guys in the community were scared of the killers as well. It's not like people in the hood saying 'Everything's cool'. It's not cool, the only difference is you live next door to the killers and the rapists, and you learn how to deal with that."

He was raised by his grandmother, who managed to instil in him the importance of not harming people unless in self-defence. Even when he was a criminal and a drug user, he says, he still held to that principle. He used to break into hotel rooms when he was a young man, and he and two friends once used a fire escape to enter a 17th-floor room at the Radisson in downtown Chicago. They found a white couple in bed. "One of my friends wanted to do something bad to the couple, to the girl, and I stood up and said, 'No way in the world are you going to hurt these people here.'" Instead he ripped out the phone, left the couple and fled empty-handed with his accomplices.

Twenty one years ago Hardiman quit drugs and drink and since then he's applied himself to being a peacemaker. He seems to command the respect of the interrupters, which – given that many of them have killed people and none has much experience of taking orders – is no ordinary feat of management. It helps that he was never affiliated to a gang, but in any case, he says: "You can't show any weakness. You show any weakness and they run over you."

What happens when someone challenges his authority? He laughs: "I've had a few guys buck up against me, but that's all right because some of us enjoy the bucking."

Although it may sound like a recipe for recidivism, the programme has had very few disciplinary problems. "We've had about 300 violence interrupters," Hardiman calculates, "and we've only had five guys lapse back into the lifestyle."

Hardiman can make statistics sound like a jazz lyric. "We mediated 351 conflicts from Jan to June 2011," he says, now on a roll. "This year homicides are down by about 30 compared to the same time last year. I'm not saying CeaseFire did it all but it played its part. So we are on pace to get under 400 homicides for the first time in over 40 years. That will be unheard of in Chicago."

Down the hall from Hardiman, Cobe Williams shares a small, bare office with a colleague. A sweet-natured teddy bear of a man, Williams is one of the stars of The Interrupters. It requires an effort of cognitive will to reconcile his benign persona with the fact that he served 12 years for attempted murder and drug trafficking. "My father was a big-time gang member," he says in a soft, half-swallowed, half-stretched Chicago accent, "selling drugs, driving a Cadillac and fancy cars. He was my role model."

As a kid (which Williams pronounces "keed") of six years, he visited his father in prison and learned how to do gang signs. When Williams was 11, his father was beaten to death in a fight. "I just wanted to rip everything up," he says. "My mother tried to do what she could but she turned to drugs after my father was killed. So I started hooking up with people in the community, mostly like me, without a father, doing all kinds of crazy things, selling drugs, gangbanging, going to prison."

When several of his friends were sent away for 50- or 60-year sentences, he realised he wanted to be the father he never had to his own son. He began doing volunteer work for CeaseFire, and was eventually hired but then laid off as a result of a funding shortfall. Each year, when the budget runs out, a number of workers are temporarily removed from the payroll. CeaseFire's funding, which comes from a mixture of public bodies and private individuals, is never enough to support its case load. Williams kept working regardless of payment. "I stayed committed because I was part of the problem," he says. "I'd fucked up and I wanted to help kids that I relate to so well because they come from broken homes like me."

The zeal displayed by several of the interrupters I met was reminiscent of the all-consuming attitude that reformed alcoholics sometimes bring to bear on the whole business of sobriety, their own and especially that of others. Williams admits that he finds it hard to switch off when he gets home to his wife and kids. It's like an obsession, he says, because he's constantly looking to build relationships on the street so that the trust is established when it comes to a crisis.

On film, Williams displays a natural empathy for people caught up in destructive situations, and never more so than in his dealings with a memorable character called Flamo, an old friend from jail. We first see Flamo in a volcanic rage, with a gun tucked down his trousers, stomping around the front yard of his house. Someone has informed on his brother and he wants revenge.

Williams tries to talk him down but Flamo, who has spent half his 32 years in prison, has no time for his platitudes. "How can you help me?" he demands. "Right now, how can you help me?"

What ensues is an extended "babysitting" job, as Williams attempts to prevent Flamo from settling the score with his enemy. I won't reveal the outcome, but some of the exchanges between the pair are worthy of the finest Hollywood double-acts, with Williams as the long-suffering straight man and Flamo as the riotously impulsive funny guy.

Perhaps the most poignant relationship that Williams nurtures, though, is with a reformed young gang member called Lil' Mikey, who is another recruit to CeaseFire. We see Williams accompany the boy, just out of jail, to the barber shop he held up in an armed robbery when he was 15. The heartfelt apology he makes for his crime is one of several scenes that lay siege to the tear ducts. "A lot of people want to change their life," Williams says, when I ask about turning young people around. "They just don't know how to do it. People always be judging them. If you keep being told you bad, you say 'OK, I'm bad'".

I meet another of Williams's success stories, a short but assertive 20-year-old called Yaya. Unlike Lil' Mikey, who is a study in contrition, Yaya maintains a street-hardened shell. "If you came here three years ago," he informs me, "I might have dropped a pistol on you, you know what I'm saying?"

I do.

Hardiman had described his accelerated youth as leaving him a manchild, and it's a designation that suits Yaya. A father himself, he tells me he's been a leader since he was eight years old, and now he is leading his contemporaries towards non-violence. "They ask how can they get involved, how can they change their lives, how can they wear dress shoes with jeans, you know what I'm saying?"

About the dress shoes, I'm not certain, but I do know that if I was kid in Yaya's neighbourhood, I'd make sure to pay attention to what he said.

Just as not all the gangs in Chicago are African-American, not all the interrupters are black. There are also a few Latinos, among whom Eddie Bocanegra stands out, if that's the right term for one of the most humble and unassuming men I've ever had lunch with.

Because the Latinos, anxious about their residency status, are much less camera-friendly than their African-American counterparts, Bocanegra isn't seen mediating on screen. Instead he is the dark conscience of the film. Just two years out of prison at the time of filming, he was shouldering an unshakable burden of guilt. Back in the mid-90s he killed a man, shooting dead a rival gang member in a reprisal attack, and served 14 years in prison. Since his release, he has thrown himself into community work – lecturing, art therapy, grief counselling and violence prevention. His coping mechanism, he tells the camera, is to keep working in an effort to try to forget what he has done. But he can't forget, and at times you can see the memory eating away at him like a psychic tumour.

The guy I meet seems much more at ease with himself. He's still busy with all his various schemes and projects, but the difference is that the remorse, while profound, is no longer malignant. "The film was therapeutic for me," he says, clear-eyed and ever vigilant to the danger of appearing conceited or insensitive. "Sometimes I thought 'Wow, I really put myself out there', but it has helped me. I still carry some dead weight but I also recognise I'm shedding some of this."

Bocanegra's beat is Little Village, the Hispanic section of Chicago. With a couple of exceptions, the city remains largely divided along ethnic and racial lines. Across town on the mostly white north-east side, on the corner of a pleasant tree-lined block, sits the offices of Kartemquin films, a production company set up at the height of 60s radicalism that prides itself on its earthy, humanistic approach to social issues.

If the company has an artistic philosophy, it's: let the people tell the story. That's certainly been the mark of the work of Kartemquin's most celebrated film-maker, Steve James, whose own motto is "No experts". He made his name with 1994's Hoop Dreams, an epic saga of hope and disappointment played out against the everyday struggles of a pair of black Chicago students, William Gates and Arthur Agee, and their fractured families. By the end of its 2 hours and 50 minutes running time, the viewer felt as if he'd intimately come to know Gates and Agee the way readers once believed they personally knew Tolstoy's characters.

This sense of having entered someone else's life made it all the more disturbing to learn that Gates's brother Curtis, who appeared in Hoop Dreams, was murdered in 2001 in a senseless argument, while Agee's father, Bo, who also featured prominently in the film, was murdered three years later in 2004 – Agee's half-brother had already been gunned down shortly after the documentary was released.

For James, a tall, rangy figure with short grey-hair and the kind of sympathetic face that must be a magnet to personal confidences, the news came as a grievous shock. He had maintained relationships with the Gates and Agee families, and he says the killings, and their dreadful aftermath, were instrumental in making him want to look at the violence plaguing the African-American community.

But it was not until he read a New York Times article about CeaseFire written by his friend, the writer Alex Kotlowitz, that he saw the means of addressing the subject. He and Kotlowitz, who were co-producers of the film, both wanted to generate a debate about violence without lecturing the audience. "I'm not an agenda film-maker, and Alex is not an agenda writer. What we really wanted to do was delve in there and understand it. You read that people in those communities become numb to violence," he says. "And that's not true. They're not numb. They're not surprised. Everyone knows someone who has been murdered and has family members in prison. We wanted people to see that in these communities, even though it's a common occurrence, it's still devastating."

For two white men in almost exclusively black districts there are inevitably issues of credibility. The access provided by the interrupters was vital, but it also helped that the pair had produced previous works of standing within the African-American community. In James's case it was Hoop Dreams; for Kotlowitz it was his book There Are No Children Here, which documented the trials of life in the Chicago housing projects. "We had this informal competition," James jokes, "which I won hands down. But actually with Alex's book, if you've done serious prison time in Chicago there's a good chance you read it. So every once in a while it would be, 'Yeah I read that book', and Alex would do like [James mimes a victory dance]."

They filmed for 14 months. Given all the death and grief they were witness to over that period, I wondered if he ever got depressed. "There was a sense of despair you'd go home with, and there are very tough things in the film that were certainly affecting. But one of my favourite TV series is The Wire, and there were days when I thought we were living in The Wire, and what could be better than that?"

The comparison is an obvious one, right down to the characters, who bear a striking resemblance to some of those in The Wire. For fans of that series, Cobe is Bunk, and Lil' Mikey is Michael Lee, the baby-faced schoolboy gangster from series 4. "Great analogy," says James, joining in on the game. "And Flamo is Omar. Except not gay. Renegade guy, not affiliated, but does what he need to do and is sharp as a whip."

If parallels with TV characters seem patronising, it's hard not to refer to fiction when the places like Englewood are so estranged from the common conception of reality. Still, there is one person who appears in The Interrupters for whom there is no fictional analogue. Her name is Ameena Matthews and she is the daughter of the Black Prince, Jeff Fort.

Although she wears the body- and hair-covering garb recommended by Islam, the faith she adopted after her father converted in prison, Matthews is sassier than Pam Grier in a bikini. She's so hot on film that she practically burns through the celluloid. Fearless and filled with righteous conviction, she confronts hoodlums and comforts the bereaved with such an extraordinary mixture of sense and sensitivity that you wonder why she isn't involved in a larger scale undertaking, like running the UN or the world.

In person she's just as impressive – quick, perceptive and teasingly flirtatious – but she also reveals a vulnerability that takes not just me but herself by surprise. "My mother was a baby when she had me," she tells me when we meet at the CeaseFire offices. "She was a baby having a baby and my dad was a baby producing a baby."

As with all the other interrupters I meet, her life is inscribed with the same narrative arc of dissolution and redemption. Completely independently of her father, whom she barely knew when growing up, she got involved in drugs, gangs and criminality as a teenager. She rose up the ranks to became a major player herself. "I hustled with the big guys," she says. Then in her late 20s she was hit by two blows. One was from a bullet, yet the one that really hurt was the realisation that she had become the person she never wanted to be, her mother. When he heard about the attack on his daughter, Fort offered to enact retribution on the assailant, but Matthews declined. In retrospect, she realised that it was her first peace mediation.

Her parents split up when she was a baby, and her mother wouldn't allow Fort to see his daughter, getting friends to say she was away whenever he called round. "He wanted to see me," Matthews says, and I suddenly notice a tear streaking down her face, followed rapidly by another. She says she has tried hard to forgive her mother but it hasn't been easy because she didn't provide maternal protection. Her mother set up home with another man who abused Matthews. "He violated me and she believed him, not me," she says, wiping away a tear. "She protected him, not me."

Matthews has since made her peace with both her parents, and she regularly visits her father in jail. "He always said to me, "Be better than me and don't end up where I am." We good now. We 98 and two. We're 100%."

All the ingredients of the misery memoir are present in these tales, yet none of them are retold to elicit pity or even compassion, at least not for the subjects themselves. For Matthews, all that matters is to break the cycle of emotional deprivation. She tells me that some of the children she deals with in Chicago have never seen Lake Michigan, the vast lake whose shoreline runs along the city's eastern edge. In the film we see her painstakingly try to introduce Caprysha, a love-starved girl in care, to some of life's small pleasures – a skating rink and a nail bar – with mixed results. Caprysha happens to call while we're speaking, and I ask Matthews how she's doing now. "A work in damn progress," she says, shaking her head, "that's my Caprysha."

Understanding violence as a disease presents a couple of manifest difficulties. What of human agency? What of personal responsibility? No one elects to contract cancer or tuberculosis in the way that someone chooses to buy and use a gun. Yet the truth is that we know there is behaviour that increases disease risk, and sometimes, in places like Englewood, arming yourself can seem less like a choice than a necessity.

If violence is to be viewed as a disease, then it's undoubtedly a contagious disease. The conflicts of Chicago's south side are not a unique phenomenon. They're also to be seen in New York, Detroit, Los Angeles and Baltimore, as well as in London and other British cities, where youth and gang killings have escalated in recent years. Environment plays its part, though you'd be hard pressed to find the segregated hopelessness of Chicago's ghettoes in even the most deprived areas of urban Britain. The germ of violence is also carried by a variety of media, such as film and music, in which image and that most precious social commodity – identity – can be labelled, packaged and exported round the world. Fashion, it seems, can be just as stubborn an opponent as poverty.

"We can change the culture," Hardiman told me, "if we downgrade or deglamorise acts of violence, make it not cool to be a violent person."

Another way of phrasing that objective is to make it cool to be a non-violent person. That won't happen overnight. It will take a lot of hard work and commitment to overcome the intransigence and setbacks that are part of any movement of social reform. But a model exists of progress in the most unpromising circumstances. For whatever anyone thinks of the interrupters, their youthful madness or their current methods, they are without question: Pretty. Damn. Cool.