Storm Catchers

A group of Indian scientists are testing out a new monsoon forecasting model, and it might just be the most important achievement of 21st century meteorology.

Storm Catchers by Colin Dadelia; Illustration by Bhavya Arora (3 Sided Coin) for FiftyTwo.in

Paradigm / Shift: Stories of innovation, shaped by intelligence.

A week before the rain began, the sky over the Bay of Bengal could have transformed into anything the atmosphere has to offer, even nothing. With four days to go, a sheet of air along the eastern Indian coast dropped in pressure. Clouds gorged on evaporating ocean. Squalls swirled and spilled over Andhra Pradesh’s forested shoreline.

It was the middle of October 2020. At the India Meteorological Department office in Hyderabad, Dr. K. Naga Ratna, the person in charge, saw bloated clouds flitting across the screens of forecasting computers. They’d formed at the very end of the southwest monsoon season and were now drifting toward her city on a mutant band of air.

Naga Ratna’s job was as much about making accurate forecasts as ensuring that the people of Telangana took them seriously. Every day of the week, she recorded brief videos about the day’s weather in English and Telugu and sent them to groups of weather reporters on WhatsApp. “Emails people may not check, but WhatsApps people check,” she said with a grin.

The media was a bridge between her office and the public. When new editors took over the weather section at a newspaper or TV station, she walked them through her process, explaining what was knowable and what wasn’t, hoping they wouldn’t print or air anything that scared people when they ought to be calm, or calmed people when they ought to be making nervous preparations.

On that day in October, the clouds in the model forecast were like grenades in mid-air. They were scattered across a stretch of low pressure so wide that they might, in theory, start to rain across a swath of Andhra Pradesh and Telangana all at once, drenching cities and farms below. But monsoon clouds had recently developed a habit of inundating small areas while sparing others nearby. Watching them, Naga Ratna knew it was going to be a week of frantic updates for government officials and reporters.

Each regional IMD office manages a digital map of districts for which it is responsible. On such maps, colours are assigned based on how inclement the weather is likely to be over the next day or so. They scale up from green to yellow to orange to red. At this point, all of Naga Ratna’s districts were set to orange. It was two days before the first downpour.

M

eteorology is a science that has evolved from biblical weather prophets to apps that declare the probability of rain at a certain hour. The monsoon is, simply, a wind pattern that blows the same way each year. Both these systems revolve around a certain order. In broad terms, they seem easy to understand. Yet to know the intricacies of either is to know that the more you zoom in, the more likely your assumptions are to be challenged. In both the monsoon and meteorology, chaos ripples in seemingly innocuous gaps; unexplained realities upend neat narratives of order and scientific progress.

Think of the Indian monsoon like a comforter pulled across a bed. In theory, this thick blanket of air would be flat and smooth, stretched taut over the landmass. But in practice there are all manner of disturbances. Oceans, lakes, and rivers release their water skyward. Heat warbles off yellow deserts and black streets, pushing air molecules apart and making more space for water. Clouds bloom like cotton ruptures. Like fate, they drift along on invisible currents.

At least since the people of the subcontinent started farming, monsoon rains have made the difference between desperation, subsistence, and bounty. Meteorologists have tried to map the pattern of these winds for roughly 140 years, because knowing whether, where and when this massive atmospheric system will release its rain would free an immense portion of humanity from its whims. Weather would still be an inconvenience, but it would no longer be a surprise.

At least since the people of the subcontinent started farming, monsoon rains have made the difference between desperation, subsistence, and bounty.

Indian meteorologists are much better equipped to predict the monsoon than their colonial British predecessors, but they still grapple with the same core question: how do clouds form? It’s one of those simple problems that leads to rabbit-holes and detours rather than a straightforward answer. One of the reasons for this is that our warming planet keeps altering the equation. The earth’s rising temperature allows monsoon clouds to soak up more water, and each year these swollen clouds unleash storms that may swamp one neighbourhood but only sprinkle the next.

These conditions have created a need for precise, short-term, hyperlocal forecasting. That’s why India is increasingly relying on a method of hourly prediction known as nowcasting. This digital monsoon model absorbs torrents of data and spits out maps of India’s winds so detailed and localised that meteorologists can see—12 hours into the future and with an accuracy of 70-80 percent [1] —how squalls might build, split, and spread over specific districts. It enables the IMD to tailor forecasts for farmer groups in adjoining areas, and to send out custom alerts for different neighbourhoods in the same city.

Monsoon models deal in terms of ‘resolution,’ which refers to the clarity with which they can picture the weather up to a certain point. The model India uses to see the entire globe has ‘12 sq km resolution,’ which effectively means that it’s good at knowing whether something will happen across a patch of 12 square kilometres but not so good at predicting where exactly it will happen inside that range.

The nowcasting model can see with ‘2 sq km resolution.’ In trying to pin down the pattern of deadly storms, it has edged weather prediction into new terrain where it’s possible to believe that, someday, meteorologists will know the route of every cloud. And yet that theoretical possibility is farther off than it seems. To achieve even 2 sq km resolution, models need to factor in how hills, streams, buildings, and highways warp the weather, but these are still just some of the more obvious elements that shape the skies. In reality, even an air-conditioner might send a cloud on a slightly different path. [2]

C

omputer models that forecast the climate’s long-term future combine a slew of equations meant to define interactions between wind, water, heat, and everything else that makes up the atmosphere. Each model churns out slightly varying predictions. Forecasts within each model change from year to year. But, overall, what happens in the computers doesn’t look too different from the real world—until it suddenly looks like no climate the planet has known.

The same equations that produce versions of the recognisable world also tend to produce simulations of a planet where the land is smothered in snow and the oceans are gigantic slabs of ice, a colder place than woolly mammoths ever roamed. There is evidence this happens because of ‘almost-intransitivity,’ a condition where a system such as the climate (real or simulated) acts one way for aeons, and then decides to act very differently. There is no identifiable reason for this, and because scientists want their models to mirror the world outside, they retooled their simulations to pretend that the possibility of Earth’s coldest era hardly exists. [3] But the Earth has known ice ages before. They’ve occurred, as James Gleick wrote in his book Chaos, at “mysterious, irregular intervals,” perhaps hinting at the planet’s other natural climate.

A major storm anywhere over India meant that Dr. Ananda Das had to spend nights in his office at the IMD headquarters in Delhi, a long, squat building just off a canopied street that stretches alongside Lodhi Garden. In October 2020, as the storm was threatening the coast of Andhra Pradesh, he was one of the few people still coming into work, trying to make up for colleagues stuck in quarantine zones because of the pandemic. The department had kept its models running like they always did, and Das watched the screens keenly as the wave of low pressure pushed towering clouds towards Hyderabad.

Das is in charge of numerical weather prediction at IMD, a frontline job in nowcasting, because statistical probability helps balance the model’s artificial intelligence. As a student, he was convinced he wanted to be a physicist, fascinated as he was with general relativity theory. Das moved from his village in West Bengal to the state’s biggest city so he could enroll in an undergraduate physics course at the University of Calcutta. But there he found that theoretical physics was nearly impossible to pursue.

“There are very few places that do that kind of research in India,” Das said, “and you have to be very lucky to be a part of them.” [4] When he switched to a more practical course for his graduate studies, the department head cancelled the class he was keen on, leaving him with atmospheric physics.

While he was working on his doctorate, Das got a job at the IMD headquarters in Delhi. As part of his research, he installed his forecasting model on the institution’s supercomputer, the country’s first such machine. He’d monitor the output as often as possible, like it was a new-born child. The engineers who kept the supercomputer running were constantly looking for ways to stay warm in that frigid room. On the nights that Das couldn’t make it back to his campus, he’d borrow a blanket from one of them and sleep in that room.

When I met him, Das said he was still doing what he did in those early days at the IMD: understanding the atmosphere through simulation. When IMD meteorologists begin to follow a menacing band of air, they first predict its path with global forecasts that give them almost an astronaut-eye view of the planet. These models are good at making broad predictions. For instance, they can help forecast whether winds swirling over the Bay of Bengal might develop into something India needs to worry about in a couple weeks.

But they can’t say which areas need to worry more than others. That question is for mid-range climate models to address, which they can only hope to do once the storms are roughly five days out.

IMD meteorologists never send a red alert for specific districts until 24 hours before they think rain is likely to fall. That’s also about when nowcasting takes over from the mid-range models, projecting 12 hours out. (Delhi headquarters issues nowcasting forecasts across the country, which are then tailored by regional offices.)

They needed people to understand the potential danger, but also had to avoid a frantic dash out of town that choked off the roads.

Das and Naga Ratna were talking constantly through the hours that led up to the day Hyderabad’s skies went dark, trying to figure out how to tell the city about what was to come. They needed people to understand the potential danger, but also had to avoid a situation where there was a frantic dash out of the city. That would choke off the roads. 

It was already clear that some areas might stay relatively safe while people elsewhere would see their streets turn into canals, but it was difficult for Das and Naga Ratna to know which was which. Even nowcasting can’t quite predict where the heaviest rain will fall, and just how heavy it will be. Meteorologists don’t get how these downpours begin, and so their simulations can hardly suss them out. The type of flooding that turns cities into inland seas is a “rare phenomenon” in the model, Das said, but in the real world it’s happening more and more.

With one day to go, it made sense for Naga Ratna to declare an emergency. On the computer screens, Hyderabad turned crimson.

H

urricanes did not crash into Texas. In the late 1800s, the science said that any swirling mass of clouds crossing the Atlantic Ocean would inevitably spin towards the north-eastern coast of the United States. In September 1900, meteorologists believed a storm was doing just that. The hurricane’s winds had slowed over Florida and were now pushing from the south. That seemed evidence enough.

A few mornings later, swells shook the beaches of Galveston, Texas, like the distant footsteps of a beast. Isaac Cline, the resident meteorologist, could hear them from his home down the street. Even then, he didn’t believe.

The storm had swallowed the city’s peninsula by nightfall. Galveston, “the New York of the Gulf,” was shattered. More than 6,000 people died. It’s still the highest-ever death toll resulting from a natural disaster in the United States.

Monsoon science has always been tied to catastrophe. Nearly 150 years before India’s meteorologists began building models that could better predict sudden downpours, their predecessors were trying to figure whether the monsoon would bring any rain at all.

Drought and the colonial administration’s reluctance to hand out cash or emergency rations had starved people across India in the late 1870s. Desperate villagers wobbled on toothpick legs in search of grain. Thousands crumpled in emaciated heaps on the side of dusty roads, their bodies like dots in a line connecting famished towns. Data suggests that six to ten million people died between 1876 and 1878 alone. [5]

The death toll was absurd, but colonial officials didn’t really acknowledge their active part in the catastrophe. Instead, a committee came together and decided that avoiding future famines was all about deciphering the signs of a coming drought. That meant extending the tendrils of empire deeper into Britain’s most important colony—even into its atmosphere. “It was the fundamental fascination of the imperial situation,” Katharine Anderson wrote in her book about Victorian meteorology, Predicting The Weather. The British had found yet another way to place “Western conceptions of order upon a vast, confusing subject.”

In 1875, Henry Blanford was appointed the Imperial Meteorological Reporter to the Government of India. His main task was to understand droughts, which basically meant that he was in charge of predicting the Indian monsoon. Once on the job, he set about bolstering and systematising a network of observatories across the country.

Until then, the people manning these stations had recorded rainfall, humidity, barometric pressure, and a host of other data using personal criteria that made it impossible to calculate averages. Some observers made up their recordings. Others took diligent notes, only for them to be forgotten in some bureaucratic basement.

Blanford’s immense task was accomplished with a supply of cheap labour made possible by racial subjugation. The government paid Indian observers just a little more than half of what their white counterparts earned, forced them to work in isolated observatories in the Himalayas, and sent them to helm stations in locations thought to be hard on European constitutions. Blanford’s mission relied on these technicians even as he accused them of performing their duties “in the slip-shod fashion that is habitual to persons of imperfect education.”

Yet the career ladder in meteorology was actually somewhat less subject to the racist hierarchies that corroded other disciplines. Perhaps because of its newness, it offered slivers of opportunity to Indians who were well-placed enough to grab them. One of Blanford’s deputies was a man named Ruchi Ram Sahni, who was entrusted with the excruciatingly public-facing job of drafting daily weather reports and sending them to the press. The two had little in common, but they spent hours at Blanford’s home talking about new instruments and the ways different climates influenced each other.

“He would often lend me new books or reports to read,” Sahni wrote in his memoirs, “and every now and again took the opportunity to impress upon me that, as a young man, I should make it a habit to read books and thus prepare myself for my lifework.”

Blanford’s office issued its first monsoon forecast in 1882. Looking at data from past years, he saw a link between snowfall in the Himalayas and monsoon rainfall across the subcontinent, but the connection was too weak to create the kind of forecasting consistency the empire sought. Droughts hardly became more predictable, and Blanford’s forecasts were scrapped. The monsoon remained an abyss for meteorologists until the early 1900s, when a young mathematician began to see patterns in data about the wind.

Paradigm Shift - Stories of innovation shaped by intelligence. A Microsoft India podcast in association with ATS Studio.

After the storm

Paradigm Shift

Listen to the first episode of Microsoft India's podcast "Paradigm Shift" - stories of innovation shaped by intelligence on Spotify, Apple Podcasts or anywhere you get your podcasts.

L

et’s say you’re a meteorologist, and you believe monsoon clouds start popping up over the Arabian Sea and the Bay of Bengal once the water reaches a certain temperature. The fun fact is that the water temperature hovers around this magic figure for most of the year. This means that even a small jump or drop in heat determines whether that year’s skies will be dismal grey or searing blue.

Now, you also know that both these water bodies are loaded with microscopic plankton, a fact that seems unrelated until you realise these little creatures soak up heat that would otherwise warm the water, perhaps just enough to hold the temperature under a certain threshold.

Scientists have found evidence that plankton might be one of the monsoon’s many triggers. Vishal Vasan, an applied mathematician who studies the monsoon at Bengaluru’s International Centre for Theoretical Sciences, said that the potential of this half-discovery makes his job infuriating, thrilling, bewildering, and maybe even impossible. Building equations to predict the monsoon’s future often seems to be about finding endless variables that could matter immensely or not at all.

“I think we have to take it seriously that the possibility exists that this system is far more integrated and connected than we had assumed,” he said.

Gilbert Walker was the mathematician who finally made sense of all the data the British empire had collected on the monsoon and weather systems beyond. When he took over India’s meteorological enterprise in 1904 at the age of 35, he didn’t know much more about weather than the average person. But his mathematical intuition meant that he didn’t have to search for clues in the movement of clouds.

After using a mathematical tool to see which elements of the monsoon affected others, he developed his own method for determining what connections were likely to be more than statistical serendipity. The Indian monsoon’s basic principles revealed themselves in Walker’s work, in turn establishing what he called “world weather,” the notion that weather everywhere affected weather everywhere else.

The country’s meteorological network kept growing after independence. New insights came to light as a result of the painstaking work of Indian scientists. One of them was Anna Mani, who aspired to get into physics research like Naga Ratna and Das, but found that the government was only offering scholarships in meteorology.

While Mani was building a career in the 1950s and 1960s, [6] the world was coming to terms with how little it understood about the Indian Ocean, “the largest unknown area on Earth.” [7] Despite the monsoon’s centrality to India’s economic and agricultural planning, the country’s vast observational apparatus had never systematically investigated the two major water bodies that surrounded the peninsula and shaped its winds.

Throughout the first half of the 1960s, 13 nations including the United States and the Soviet Union sent a fleet of ships and a small air force to gather data across the Arabian Sea and the Bay of Bengal, in a mission called the International Indian Ocean Expedition. (Colin Ramage, the New Zealander in charge of Bombay’s International Meteorological Centre at the time, remembered being strapped into one survey plane as it rattled and bucked through mushrooming monsoon clouds at 20,000 feet, at one point plunging the length of a football pitch “in a single second.” [8] )

New weather data flooded fax machines at the small Bombay office that served as the expedition’s headquarters. Staff translated pages of Morse code and jumbled numbers into facts about India’s weather patterns. But the expedition’s most significant find hinted at changes to the planet as a whole: there was evidence that the Indian Ocean and the atmosphere above it were taking on an enormous amount of carbon dioxide, altering the chemistry of water and air. Only a few recognised the implications. At the time, the idea that people were overheating the planet seemed a problem with consequences too distant to consider.

Roughly around the time of the expedition, American mathematician and meteorologist Edward Lorenz made his own momentous discovery, which showed that the weather’s innumerable complexities made it impossible to predict beyond two weeks. There were simply too many variables interacting with each other in too many ways. [9]

Meteorology was making progress with long-term monsoon forecasting, but here seemed to be proof that more granular efforts had an impassable ceiling—and that Indians would always be at least partially beholden to the whims of seasonal winds and rain.

Over time, as rogue squalls have begun to swamp small patches of ground with little warning, hour-to-hour predictions have become the kind that can save lives or give farmers enough time to rush out and protect their crops. The IMD can’t see these storms even close to two weeks ahead of time, but the goal of nowcasting is to edge toward Lorenz’s limit.

I

 play basketball at an outdoor court in Bengaluru, which is the only reason I’ve ever used my phone’s weather app. I used to check the chance of rain a few hours before I left my apartment, but it didn’t take long for me to work out that the app was useless. At least three times it told me there was a 100 percent chance of rain for the next 12 hours, but then not a drop would touch the court. 

Weather apps source data from models that are pretty bad at determining what’s going to happen in the sky over one person’s particular head, which leads to rain predictions that aren’t very relevant. Maybe the app was telling me that some part of Bengaluru was going to see rain in each of the next 12 hours, but anyone who lives here knows that rain over one part of the city can seem almost irrelevant to whether or not it rains over another.

When Vishal Vasan was doing research at the meteorology department of Pennsylvania State University, where forecasting company AccuWeather was founded in the early 1960s, he always pulled out his phone to check their app before walking home, ten minutes away. If the app told him it was going to rain any sooner than that, he waited.

“They will tell you it’s going to rain in seven minutes, and they mean seven minutes,” he told me. Accuweather has such total grasp of campus weather that they can track individual clouds across the sky and tell you exactly when they’ll pop. Satellites over the area see everything, and there are almost never any sudden changes that push a storm off its path.

Physics works differently close to the equator. The Earth’s rotation is far more pronounced the closer you get to the poles, so weather patterns change more slowly and are easier to predict.

This kind of forecasting is possible in India, but zooming in to that level of precision is fundamentally more difficult over, say, Bengaluru than it is over Pittsburgh, for at least two reasons: one theoretical (physics) and the other practical (limitations in data gathering).

Physics works differently close to the equator. There’s more sunlight and a lot more water, which means that heat and moisture distort weather patterns in ways that would never happen in North America or Europe. The Earth’s rotation is also far more pronounced the closer you get to the poles, so weather patterns change more slowly and are easier to predict. At the equator, for the purposes of forecasting, the Earth might as well be an unmoving rock.

There’s also never enough data. India’s institutional network is vast, but when more elements affect the surrounding weather, more things need to be measured. Vasan told me that if meteorologists want to forecast monsoon clouds over a single neighbourhood, they will have to know things that sound absurd to even consider, such as how much moisture steams off streets, sidewalks, and patches of grass. (The figures will be different for all of these features even if they’re lined up next to each other along the same stretch of road.)

“That kind of measurement is an insane idea,” Vasan said. “Nobody in the world does this.” Ultimately, meteorologists get around what they can’t measure by approximating. That’s how they build models. If they don’t know how much moisture wafts off a particular area, they tap in a few numbers as a guess, test the model against the real weather, and then adjust it the best they can.

This works well for long-term monsoon forecasting, because approximations don’t have much effect on broad outcomes, but for short-term forecasts over much smaller areas, all that rounding spins the models into some deranged alternate universe within a few hours.

The benefits of stretching towards Lorenz’s theoretical limit are obvious, but given the amount of time and energy it takes for small improvements, doing so risks becoming a vanity project. Sudden floods are killing people and flattening crops in the here and now, so there is hardly time to determine exactly which district will drown and which will be spared. That’s why the IMD focuses on probabilities: there’s a 90 percent chance this district will flood, but only a 60 percent chance the neighbouring one will. With numbers such as these in hand, officials can use their limited resources to prepare infrastructure and people for what is most likely to come.

W

hen smoke and car fumes drift skyward, they can blend into clouds. Every microscopic particle of pollution is a surface that might reflect a small ray of sun or on which a bit of moisture could condense into water. Enough fumes, and these mutant clouds can shift the temperature of entire regions, or change the nature of storms.

Blankets of smog suffocate swathes of India, home to 22 of the world’s 30 most polluted cities. [10] Deepti Singh, a monsoon climate expert at Washington State University in Vancouver, told me that monsoon models know this affects cloud behaviour, but they don’t understand how smog influences specific squalls. 

“There’s a lot of uncertainty there,” Singh said.

The road up to the IMD office in Hyderabad peels off a thoroughfare and ambles through overgrown fields. In a couple of minutes’ walking, even the honking is hard to hear. Birds chirp. Squirrels potter across the asphalt, unafraid of a road that gets little traffic. Above the trees is the only tall building in sight, topped by an orb that’s painted the light grey of a smoggy moon and houses a giant revolving satellite dish.

On the afternoon of 13 October 2020, Naga Ratna undertook the opposite journey on her way home. She had been in the job for a year by then, long enough to know that what happened over the next few days could determine the immediate future of both her career and her city.

“The lucky thing is that everything happened after office hours,” she told me. She got home before the streets became too clogged to pass. The rain, when it came, was the kind that even nowcasting models struggle to predict. It swallowed the city in chunks. Car roofs bobbed along former roadways like flimsy rafts. Entire homes collapsed onto families huddled inside.

The skies ran dry after two days, but the rain returned on 17 October to lash the eastern parts of the city. By the time the water receded, at least 70 people had died across Telangana. Naga Ratna’s forecast memos had earned the trust of the chief minister, but there was nothing she could do about the lack of resources and preparation needed to get a state through an onslaught of this sort.

After the storm, Naga Ratna focused her energies on developing an additional layer in India’s meteorological architecture: damage forecasting. Its objective is to identify the most vulnerable parts of a district in order to minimise the carnage that storms inevitably bring. Damage forecasting finally integrates meteorology into the country’s web of disaster response. It refutes the discipline’s colonial origins by turning away from precision and moving towards preservation.

Epilogue

I

n September 2021, Naga Ratna called me to say she was going to be late. Her son had just secured a seat in an engineering college, and she wanted to enjoy a celebratory moment at home. 

Her staff filed into the room one after the other as soon as she arrived. “I’ve come later, so everyone wants to brief me,” she chuckled. The phone on her desk went off all day, sometimes ringing at the same time as her mobile. “We are on phone calls around the clock,” she said.

Naga Ratna’s department is in charge of Telangana’s 33 districts, each with their own topographies and infrastructure. They’re home to mountains and riverbeds, coal mines and power plants, packed cities and sleepy towns. Rainfall presents a unique challenge to each of them. For months, she’s been collecting details about terrain and infrastructure from local officials. Her colleagues upload this information into a system run by headquarters in Delhi, which then plugs the metrics into damage forecasting models.

Naga Ratna uses the findings from these models to tailor bulletins for a wide variety of interest groups and officials: agriculture officers can warn farmers to lug their drying crops inside before a downpour spoils them; doctors can make sure backup generators are working in case the power goes out; police can take a call on which roads to block so people aren’t caught in low-lying areas. Everyone, ideally, will have a plan that gets ahead of the storm.

Dusk dropped quickly over Hyderabad on the evening of that September. A smear of clouds smudged out what was left of the sun. Thin streaks of water fizzed off the main road near Naga Ratna’s office. The drops began to plump, slapping asphalt and concrete, and suddenly half the people outside were pulling out umbrellas. The shower stopped in something like 30 minutes, just enough to steam the city. The umbrellas were tucked away again. The rain had been unremarkable, just something everyone dealt with while going about their day.

Colin Daileda is a freelance journalist in Bengaluru, India. He has written for The Atlantic, The Washington Post, The News Minute, and many others, and now often writes about climate change and environmental degradation. When he is not doing that, he is maybe playing basketball or eating a doughnut.

Acknowledgements: 

This story would have been much diminished without the in-depth and meticulous research of the several authors mentioned in the text. Their books allow us to understand the strange, convoluted evolution of weather science, and therefore how this evolution affects our understanding of the weather we see today. 

Thanks very much to anyone and everyone who took the time to explain the math and science behind monsoon prediction. Any errors, I'm sure, are mine alone. 

This story is part of Paradigm Shift, a multimedia series exploring innovation shaped by intelligence, brought to you in partnership with Microsoft India. Microsoft India exercised no editorial control over this reportage.