This is going to be about what this community of data science,
big data, data mining that has revolutionised
the way we do commerce, live our daily life, entertain ourselves,
elect presidents, and so forth.
What could it do for the world of climate?
The domain of climate, like most of their domains,
used to be extremely data-poor,
but a slow transition started happening about 50 years ago
with the advent of satellites and then with faster computers
and more sophisticated models.
You see a realistic simulation of Earth's systems model
on the top left of an NCAR.
There's a massive amount of data being generated,
literally petabytes of data generated on a daily or weekly basis
and leading to exabytes and beyond.
As a result, there is a great realisation in the climate science community itself
that they, of course, have always been working
with data for all the time,
but they realised that the climate research
has become big science comparable in magnitude, complexity, and importance
to human genomics and bioinformatics, which went through the same transition
starting about 30 years ago.
This is beginning to happen very rapidly now.
What it means really is that this community really needs
the help of all of the different techniques
that we are able to bring to bear on a wide variety of problems.
I am part of this project on understanding climate change
using data science approaches.
This project, of course, half the team is a computer scientist,
half the team is people from other disciplines,
Northeastern, NC State, Northwestern, and others.
The objective of this project is to see
what are the different techniques in data science that we bring to bear
to really impact some of these climate science applications.
This slide is just to give you a very broad overview of different things
that are going on.
On the top left,
you see the study of the oceans using these ocean altimetry data,
the satellite altimetry data for the last multi-decades
to study the dynamics of the oceans' currents and eddies.
The top-middle part is showing you
how the climate systems can be converted into a network
and then eventually it could be analysed just like any social network.
The techniques that you develop to find teleconnections and relationships
then can be applied to other data sets like the brain, the fMRI data sets.
You heard some of those talks yesterday in the yesterday's keynote.
Then, of course, the list goes on.
I'm just going to talk about today on one of these topics of change detection
using satellite data looking at the land surface change,
which is the middle-bottom circle here,
and I'll have something more to say about it.
But from this very wide range of problems that you see here,
one thing that's common is they have lots of data.
We talked about that.
We say, "Well, we must be right to address these challenges."
But the big problem is that the data sets that we have to deal with in these domains
are substantially different than the kinds of data set
that we are used to in our community because these data sets
about the observation of a physical phenomena,
which is often sensed and observed
at multiple resolutions and multiple scales from sensors
that are embedded either in the space, which are doing a gridded data set,
or could be sensors, or could be balloons, could be buoys in the oceans.
There is a huge difference in the way we look at the data
when you're dealing with customers and the social networks and so forth
as opposed to in the climate domain.
There's a huge amount of variability.
Of course, variability exists in any phenomena.
We're not, in this community, unaware of this.
But the amount of variability
and a different timescale that it happens like in weather.
There are seasonal variations,
there are daily variations, and there are decadeal variations,
and they all combine.
The phenomena that you're trying to understand
could be the predictive model that you're trying to build.
The relationships may be very different
in different locations on the earth and during different times.
There's a spatial and temporal heterogeneity,
which makes the job very complex.
As usual, because of the way these data sets are collected,
there's often a lot of noise and missing values,
clouds and all sorts of issues,
and lots of data, but very little ground truth,
and the phenomena of interest is of course very rare.
These are all of the issues that of course
both make it very challenging and also very interesting
within the process of addressing these challenges.
Then we are forced to develop new techniques,
but then those techniques, those methods are not limited
only to climate.
They can go and be applicable elsewhere.
As I promised to you,
rather than talking about the entire giant problem,
I'm going to focus on one subset of it,
which is the earth being observed from the satellite, from the space,
using instruments that are providing a wall-to-wall coverage of the earth
on a temporal and then spatial basis.
What you're seeing here in the animation is the animation of a MODIS instrument
which takes picture of every single location on the earth
every single day.
It's a very coarse picture.
A square kilometre has about 16 pixels.
These data sets happens to be available for the last 17 years,
starting in February 2000,
available to the entire world free of charge.
This is just one example, LANDSAT, which actually went up 40 years ago.
But this data set became available only recently
as much finer spatial resolution, but not as good temporal resolution.
That's available.
Europeans have launched a very high-quality satellite,
a series of Sentinel-1, Sentinel-2,
and more coming that provide even finer resolution data set
at 10-metre resolution, again weekly or five-day coverage
of the globe.
Then there are all sorts of smaller projects have come up
in which people would launch CubeSats, satellites of this size or literally…
Throw them in the sky and they will run around for about a year
and then provide very high-quality pictures.
There are all sorts of data sets available.
Now, the size of these data sets can be very large.
Just 10-metre resolution and Sentinel,
for example, would be about a trillion pixel
for the entire globe.
Each pixel, you're collecting about a dozen different variables.
This tells you that there's a lot of data.
You can collect this data into spatial mosaic
that you're seeing here on a temporal time series
at a linear location and it offers a whole bunch of opportunities.
I'm just going to highlight three different examples
of which the first two I'll just talk very briefly
and try to give you a little bit more of a detailed presentation
on the third item.
The first two, and again, in each one of them,
I'll try to emphasise the challenges
that are faced in addressing these problems,
the novel methodologies that got developed,
and the implications of those novel methodologies
and the broader data science.
One of the first problem is being able to map the forest fires.
Why is this important?
Because the population is increasing.
We are seven billion people now.
We'll be about 11 billion people not too far into the future.
The food needs are going to go up.
We need more food, but we have less land.
What's happening right now is a lot of these forests are being converted,
especially in the tropical area.
They're being burned and converted into plantations.
To be able to monitor those and make sure
that those destruction of forest doesn't happen,
we need to be able to monitor these forest fires in tropic,
which is very hard thing to do,
and being able to map which of these locations
are being converted into plantations
because most of the world's largest corporations
have promised not to buy products that come from the supply chain
that leads to these forests that are destructed.
But then they really need to know where their products are coming from.
All of this involves solving this problem one and two.
Now, solving the problem number one
happened to lead to an entirely new formulation
in which you build a predictive model
and you don't have ground truth of the kind that we're used to.
I'll very briefly talk about that.
Why were we forced to develop a model like this?
Because there is such a heterogeneity in different part of the world
that if you try to build a relationship between what you see in the spectral space
and whether this area was burnt or not,
this relationship changes dramatically over different locations.
If you build a map for California, it won't work in Halifax, for example.
If you build in North America,
definitely won't work in Africa or Indonesia.
The phenomena that you're trying to observe is highly imbalanced.
Most of the forest mostly remain intact, only a few of them are burnt.
All of these challenges that come together
make it very interesting and very difficult to solve this problem.
It took us about seven years to correct this
in a very satisfactory way.
We have to use a phenomena that is for these problems,
even though the gold standard ground truth are not available,
but poor quality labels are available
for every sample in terms of other variables
that can be collected to satellite, for example, the thermal anomalies.
Using that, it becomes possible.
A whole bunch of innovations that have been done in this community,
in data science community, going back 20 years,
many of them are built in the context of working with imperfect labels
in annotating annotations using the mechanical torques and so forth.
That led to a formalism which now is able to build very accurate maps
of forest fires in the tropics like in the Amazon and in Indonesia,
and is far superior in quality and coverage
to the state-of-the-art product.
The second thing I talked to you about
was the mapping of plantation dynamics in tropical forest
because these forests are burned and then eventually they're converted
into plantation in many cases.
The phenomena here happens to be something that's not a discrete phenomena.
Fires can still be considered more of a discrete phenomena.
The forest was intact and got burned.
Now, in the plantations,
you plant the trees and then they slowly grow over
and eventually they look like forest.
You have to capture the entire process.
It turns out that some of these deep learning methods
with substantial extension to include spatial information and other aspects
were very powerful in handling this problem.
Especially, the problems that exist here
is that the ground truth that exist given the importance of this problem,
there are a lot of international organisations
who are trying to build maps
of what areas are plantation and what areas are not.
For example, there's organisation called Transparent World which has a map in 2014,
one-year map, but it happens to be highly inaccurate.
There's another organisation called RSPO which has a map,
but only for a couple of years, and that happens to be very incomplete,
a little bit more accurate.
Using these new methodologies,
now we're able to build the entire South Asia maps
of the plantation dynamics.
If you look at this picture very closely,
you will see that the red areas were the plantation in as of 2000.
Each year after that, 2001, 2002, the right one, four, five,
you see how the plantation have been growing.
Then you have a very precise idea
as to which of these plantations were grown,
built in which year, and using the previous project,
you can figure out which of them were burnt
and which of them were burnt and caused collateral damage.
All sorts of questions can be answered that are very important.
Again, these two are examples that you can handle these challenges
and in turn, come up with very interesting algorithms
that have a much wider applicability.
In the case of these RNNs for monitoring the forest and plantation dynamics,
now, extended version of these ones are being used to now map crops
at 10-metre resolution using Sentinel data
and providing much better quality results than anything else that could be obtained.
Now, let me just talk about a third case study,
which I'm going to spend a lot more time on,
which is being able to map the dynamics of surface water.
Now, of course, this resulted in a whole bunch of new innovations,
things like ensemble learning that can deal with heterogeneous data,
physics-guided learning, I'll talk to you about it a little bit,
and very interesting concept
about how do you transfer information across space and time
because these phenomena are observed
at different spatial and temporary resolution.
Is there a methodology
that you can leverage multiple types of data
to get better results?
Why is the monitoring of water important?
Of course, we as a humanity cannot survive without water
and our actions impact the availability of water.
You see, Aral Sea in 1960s,
it was a big giant lake, the largest lake in Asia.
More recently, it has become a tiny sliver.
On the bottom, on the left side,
you see Cedo Caka Lake in Tibet in 1984, a frozen lake.
In 2011, it's all melted water.
Basically, human actions and climate change
is impacting water supplies.
One of the biggest,
perhaps the first major impact humanity would see of climate change
would be through water in many different forms.
Being able to understand what's happening to water,
the water cycle, the flow and stocks is extremely important.
It turns out that for this problem,
one could say that we have a massive data set available for training
because in February of 2000, given the importance of this problem,
the NASA launched a special mission of the space shuttle just to map water.
For one day, we have a reasonably good…
It's not a perfect map, but a reasonably good map
of the entire globe of where the water was.
Except that it tells you on one day what happened.
You need the dynamics.
But you could potentially use this as a starting point
to build a predictive model, except that as you build these models,
you have to worry about a whole bunch of things.
One of them is, just like anything in nature
and in the climate and ecosystem domain,
there's a huge amount of heterogeneity in space and time.
What you're seeing here in these pictures are the multispectral composites.
This is like saying you have the multiple spectral data,
multiple spectral bands, you map them to red, green, blue,
and then you can see them.
All of these pictures are built using only one mapping,
but the colours tell you how heterogeneous things are.
You can see even in a small region of Africa, one continent,
you can see multiple lakes
and each of them looking very different in terms of water and land.
Visually, when you see it,
you know where is water and where is land.
The darker portions are water.
You can see that they all look different in the spectral space.
There is no one region of this multidimensional space
which is this is water and this is land.
Then the lower portion is showing that even at the same location
in Argentina, same lake,
on two different days, it could look very different.
Then you add to that all sorts of other issues
like the clouds and the noise and then the mountain shadows
and all sorts of things, and the algae in the ocean, in the lakes,
it becomes a very difficult problem.
The other problem is the data quality.
When you're collecting stuff from the satellite,
things are very hard to see.
I was giving this talk in Beijing, so I picked up this Poyang Lake from China
which supplies fresh water for the city of Beijing.
Anytime you see the red colour, which means data is not there.
If you see this movie, most of the time, data is not there.
The question is, how do you work in situations like these?
Now, this to begin with might seem like a challenge,
but as I've said, this is impossible problem.
Let's move on to something more easy to work with.
But then struggling with this
and trying to figure out how do you really solve this problem
can actually lead to many interesting solutions.
Being able to deal with the heterogeneity aspect
led to novel solutions for dealing with heterogeneous subgroups.
Again, papers and different conferences in this field.
But then these methods are also being used in health care now
to figure out how do you build models
that will be applicable to only different subsets
of the population.
This is one innovation in ensemble learning method
of heterogeneity.
The other one,
which actually is almost like a gift that keeps on giving,
and I'll talk about it a little bit more, is something that have come across
in any of these scientific domains
where something that we don't use in this community
becomes extremely important, which is typically, we work with the data,
you give me the data, you give me the labels,
I'll build a model.
But there's a massive amount of information about the domain
that can be very useful.
In this particular case, one thing that we know
is that water bodies happen to form in cavities,
in the physically relevant cavities,
which means if location A was labeled as water
and location B was labeled as land, then you know there's something wrong.
Now, this is, of course, a trivial observation,
a very high-level observation.
If this elevation information was available perfectly,
I could use it to improve any classification algorithm.
But it turns out that on a global scale, either the information is not available
at the right resolution or it's of very poor quality.
The biggest challenge becomes one of,
how do you use this insight without having the elevation data?
There's a physics, but I don't know it.
This is like implicit information about elevation needs to be used
to figure out how do you make these corrections
and how do you transfer information across space and time.
These collection of ideas, per few,
actually have allowed us to build a system.
If you're sitting in the audience,
you can go to this link and type it on your iPad or your laptop,
and you will be able to now see the dynamics of the entire globe
for surface water at a level that has never been made possible before.
Then, of course, the system is public, anybody can see it.
These always several months behind our latest results,
so it won't have all the things that I'm going to talk about,
but you will be able to see many things just by going to this.
It basically maps dynamics of all major surface water bodies
that are at least 10 pixels large, a modest 500-metre resolution,
that means about two and a half square kilometre,
which means about literally tens of thousands of water bodies
that you can see.
I gave you one example of melting glacier lakes in Tibet.
You can find all of them.
Rivers change course,
and rivers change course because the silt flows through,
as you deforest the trees and the tropics,
the silt goes to the rivers and then they end up changing their courses.
A lot of focus on building hydropower dams
in response to our actions that we wanted to build renewable energy.
The energy for water was supposed to be the cleanest and the best.
It turns out that has huge ecological impact, negative impact.
In the US, all the dams are being broken and rivers are being decommissioned.
Hydrologists want to study what these dams are, where they are,
and what impact they have using the system.
I'll show you, you have a global picture of what's happening in the last 17 years.
The list goes on.
Since I just have a few minutes,
I'm just going to show you very quickly a few examples
to give you a sense of what the system looks like.
If you went in and you clicked on a certain tiles,
the circles will show up.
If you click on one of those,
this happens to be the Don Martin Dam in Mexico.
The system would allow you to tell you how large this surface water body has been
in different times, going back to 2000.
You can see this body was 400 pixels in the beginning of 2000.
Then it would be shrank, then it became big, then it shrank,
became big, and so forth.
This just happens to be normal variation.
How do you know this is really good?
You can see pictures on Google Timelapse.
The system would let you see
the time-lapse images wherever it's available.
You can see that this image is dancing around in the same way
as you see in the time series.
You can colour these dots in terms
of whether they are increasing or decreasing in the last 17 years.
In this picture,
the greens are shrinking and reds are increasing.
You can see the examples there.
Then you can see a whole bunch of location
and vast area of South America and Argentina.
Everything is green.
Very suspicious.
Well, this area had a lot of water back in 2000, 2003,
but lately, it has very little water.
Every single location in there,
every single water body has this phenomena.
That is, you look at anything,
it grew until 2002 and then it shrank very rapidly.
It happens to be a farming area.
There must have been some interaction
between the human actions, farming, and the water supply there.
I talked about Tibet.
When you look at the system
and you come to Tibet, you will see all these red dots,
all water bodies increasing.
If you go zoom into this,
it will give you the little areas
which are specifically the area that changed,
increased over the last 17 years.
You see another area that increased here.
You can again go to timelapse and then you see all of these changing
over a period of time.
If you look at the last 17-year period,
the total amount of new water that has shown up
is of the order of 8,000 square kilometre.
New water bodies have showed up in Tibet.
Now, this area supplies water to rivers that feed about two billion people
on the planet.
This is a very sensitive area.
Any implications of climate change to this area are very serious.
Before this, people would study these lakes on one day at a time.
That is, somebody would go to one area and use satellite images and say, well,
I have a map of this area, how these lakes look like on a single day.
Now, you have a continuous coverage of everything.
You can see river meandering.
For example, when you see the red and green dots nearby,
you see rivers meandering, and then you can see them.
Again, all of this is available on the system.
I'm just giving you examples that you would be able to see on your own.
You can see island shrinking in response to rising oceans.
You see that little blue area
on the surface of a small island of South America.
This is the area that got shaved off in the last 17 years.
Now, I'm hoping that this time-lapse image…
Okay, now time-lapse image for this one is not working for some reason.
But on the system, you may be able to see this little island.
On the system, you'll be able to see the island shrinking
even from Google Timelapse.
I told you that you can see the construction of dams.
I was giving this talk in Tokyo
and say, "Look, here's a dam that was built in 2006."
Of course, you can see that because it's area,
there's a water body that wasn't there before 2006
and it came back to something bigger.
But the major utility of this
is that the hydrologists who study these dams all over the world,
they have a database called GRanD, Global Reservoir and Dam Database.
It was built in late 1990s.
It was pretty accurate until then.
After that, it's simply relied upon people reporting when the dams are built.
These are the dams that are reported all over world for the last 17 years.
This is what our system finds.
Of course, you find all the usual suspects,
China, India, Brazil, Middle East.
A lot of these places have built a lot of these dams.
What I'm saying is that you have this thing
that's available to hydrologists for very quickly.
All of this is actually now leading to a point
where we can really provide a lot of actionable information.
I'm just going to talk about two major actionable information
that this project is beginning to lead to.
One of them is there's a great interest from the international community
in trying to figure out how much water we have stored
in our reservoirs around the globe
and how these water supplies have been changing.
Water storage has been changing
over the last several decades in response to climate change.
Now, we have the surface area for all of the water bodies
that are large.
For many of them, we have the altimetry data
from the satellites.
Combining the two, we are able to provide this information.
We just call one of the essential climate variable
and our group is responsible for providing this.
The second thing that's extremely important
is being able to measure the flow of water across rivers
because rivers tend to run through different countries.
There are agreements on this to how much water you're supposed to use
and how not to supposed to use and how the water supply may be changing.
Countries are very protective of that data.
Even within a country,
different states would not share information
about how the water is going across.
Now, what this system effectively allows
is being able to construct that information
on an automatic basis on a global scale.
It's not direct.
There are a few other steps that are needed,
and those steps are needed for the third point as well.
Third point talks about being able to calibrate hydrological model.
Hydrological model may seem something very fancy or technical term to you,
but there's a national water model that's being constructed in the US
which is supposed to predict the stream flow
for about two and a half million locations for the entire country.
The idea is that wherever you live, there could be a flood,
there could be a shortage of water if you're a farmer
and you really need to know what it is.
That model is going to be calibrated using data only from about 4,000 locations.
Given the heterogeneity in this space,
people are worried that this model would work extremely poorly
because the world is far more heterogeneous
to be captured by 4,000 locations
when you're trying to predict two and a half million locations.
You need much more validation data.
It turns out that by being able to provide very precise estimates of surface water
and how it's changing, we can solve that problem.
But for the last two examples,
what is really needed is much higher resolution maps
of surface water.
What I showed you and what you will have on the system
is maps at modest resolution with 500-metre resolution,
and those effectively are at daily scale.
But actually,
they're effectively at eight-day scale because daily data is very noisy.
Now, what we want is a much finer resolution maps
and hopefully on a daily scale.
That requires being able to transfer information across space and time.
Again, I have only about a minute, I assume,
so I won't really try to go into the details of this.
But a series of technology,
a series of methods got built to transfer this information across space and time
that make use of the physical notion that I talked to you earlier
and that has resulted into…
I'm just giving an illustration of a very preliminary version,
but you're seeing here is a reservoir in Afghanistan, Kajakai reservoir,
it's a big reservoir.
You see on the date on the top, July 24th, 25th, 26th,
it's a daily observation that you're seeing for about two weeks.
On the left side,
you're seeing its picture constructed at the MODIS scale.
MODIS scale means each square kilometre got broken into four pixels.
They're pretty large pixels.
They're square pixels.
On the right side,
you're seeing a very fine image of the same reservoir being constructed.
All done using this information transfer scheme
that I have just barely mentioned to you and not told you about.
One thing that you can see
is that the right image is very fine-grained.
Of course, you don't know whether it's any good
because I could just show you anything.
To make sure that I'm not showing you something that's irrelevant,
I'm going to show you the image for one date
for which LANDSAT information is available.
This is the date.
What you see in the background
is the image from LANDSAT satellite, LANDSAT sensor,
which is a 30-metre resolution, very fine resolution.
When you look at the LANDSAT sensor,
you can see the black areas are water and then the non-black areas are land.
You can see on the left picture, the MODIS is grossly simplifying the image
because if you look at this bottom part right here,
it's taking this entire area and calling it a single line
as opposed to looking at these fingers that are protruding.
Now, if you look at the right image
which was constructed using this information transfer scheme
without looking at the LANDSAT data,
you can see that it matches LANDSAT image much more carefully.
This is the thing that I'm talking about,
new innovations that can be brought back to much wider range of applications.
I'm out of time, so I guess this is going to be my last slide.
I hope I've convinced you that big data in climate offers
great opportunity to advance our understanding of our climate.
Very important for society.
But it also offers opportunity to advance our field
because each one of these techniques
that are developed here then have much wider applications.
The other message, again, coming from this experience
and many others that I like to be able to share,
is that in many of these scientific domains,
there is a lot of knowledge that exists
about the applications and the physical phenomena.
We need new methods, new techniques, new approaches
to bring them into data science approaches.
Of course, once you do that, hopefully,
they'll be applicable in much wider set of domain.
With that, thank you for your attention.