colsto · Transcript · · Updated
This background sound leaks your exact location – Transcript
Video by colsto · Watch the original on YouTube
Summary
AI-generated from the transcript. Verify details against the source video.
- Colsto demonstrates a geolocation technique by using airplane Doppler curves from background audio and ADSB flight data.
- Colsto applies filters for time offset, velocity, distance, and silence to narrow down candidate locations across the continental US.
- Colsto matches modeled Doppler curves for actual flight geometries to score candidate points and successfully identify a security camera's exact location.
Use this video with Pro
Reading, copying and transcript downloads are free. Create saved AI results for this video with a signed-in Pro account ($9/month).
Transcript
Search ignores accents. All transcript text stays visible.
I have found off-grid structures in the middle of a rainforest, things 600 meters deep in the ocean and some stuff I can't even share on YouTube. But what I'm going to share today is no exaggeration the single most powerful geoloccation clue I've come across. And to illustrate this, I'm going to geoloccate this. It's a video, but nothing really changes visually. If you want to pause and try it yourself, it's linked in the description. The only information you have is that it
was filmed September 8th somewhere in the continental US. That's all you get. But with that information, it is possible. And I'm going to show exactly how. So, I did a video geoloccating with thunder sounds. The benefit is that they travel really far and they penetrate structures extremely well. The downside is that we were only able to get about 5 km precision. that could tell you basically what city it was filmed in
and maybe what side of town. What we're using today has those same benefits, but here's the precision. So, I'm going to play you two sections of audio. One has the information required to find this location. The other does not. See if you can tell which one does.
You probably couldn't tell because it's completely inaudible, but if I open this in a spectrogram here, we can see all the frequencies. So, if you're not familiar with a spectrogram, the xaxis is time and the y-axis is frequency. Now, even with this, you still really can't see anything. It's only when I zoom into the right frequency and then increase the resolution that you start to see it. I'll do one more.
This is a Doppler curve from an aircraft, and it's what we're going to use to locate this video. Now, fair warning, this is insanely powerful, but it also requires a lot of complex stuff. This took me months to build, over 5,000 lines of code. I was working with 50 GB Park files, 3D physics modeling, countrywide DEMs, and so much more. That's all to say. This is going to be a long video, but I'm going to explain every step in a way that anyone can understand it, even if you
know nothing about any of this stuff. So, with that out of the way, let's look at the rest of this audio. We have one Doppler curve here. If I keep going to the right, you can see another one. It's a lot louder, and you can also see the harmonics on top. I'll zoom out a bit. I think we have one more here. So, these two are what we'll be using. But, let's take a step back. Why do we even care about Doppler curves? What are we going to be using these for? So, there's something called ADSB data, and it's
data on pretty much every plane in the sky. Its position, speed, altitude, and a bunch of other things. So, my first thought was, could I narrow this down to only places that heard a plane, and then 920 seconds later another one? To do that, we need all this ADSB data first.
They have an API you can pay for. There's also a Python package just to scrape the data, but it's massive. So, I ran this for a few hours to get all the flights in the continental US on September 8th. Now, here's what that data looks like. By default, it downloads the entire flight path. That's why you see things going to Hawaii over here or other countries. But anything that flew over the US on that day is in this data set, including the two planes in our recording. So, now we just need
to find them. So, like I said, I want to narrow down to the locations that heard a plane and then 920 seconds later another one. Now, these Doppler curves are made by a plane, of course, but specifically a propeller plane. I boosted some frequencies so you can hear it better. Let me play you a bit of the louder one, and you can hear pretty intuitively that it's a propeller plane.
One of the fields in the ADSB data is type code. Now, this doesn't say propeller versus jetplane, but I did some research, found this lookup table, and this does. Now there's some missing data in that lookup table. So instead of filtering to the propeller planes, I just filtered out all the known jet types. But after doing that, the map looks like this.
But now time for this time offset filter. So I'll do this on a small area to illustrate first. This pink line is the ADSB data for one flight. You can see the individual flight points here where we have coordinates, altitude, time stamp, things like that. What we do is lay a 200 meter grid of points and then for each point note the time that it would have heard this flight. Now it's time for our first new terminology.
CPA, not that one. Closest point of arrival. So for an observer, say here, CPA is where the plane is closest to that observer. The reason that's important is when we say heard a plane 920 seconds later, hearing a plane is not really a single point in time. If you're standing at this point, you hear the plane here and here and here. This whole Doppler curve is almost 2 minutes.
So, what we need to use is the CPA time, which is roughly the middle of this. It's when the plane was closest and loudest. So, for this flight, lay a 200 m grid of points. I did a pretty conservative 10 km radius. Like, beyond that, you probably wouldn't hear a plane even in like ideal conditions. So, that's just make sure we don't miss anything. And then for each one of these points, we note the CPA time for this flight. Two important things here. First is that CPA time actually means the time
when the plane was closest. For this example, that would be probably this point. So 22 46 36. That's actually not what we need, though. Now, for the second terminology, what we need is the delayed CPA time. It's the time that the CPA would have been heard by an observer. And that's because the sound has to travel from here all the way 3 plus kilometers to a listener. That was the first important thing. The second is
that planes are in the sky. Meaning the distance that we use in this calc needs to be the slant distance, not just the horizontal distance. Luckily, the ADSB data gives the plane's altitude. However, altitude is not the height above the ground. Altitude is the height above sea level. So to get this slant distance, we actually need to know the elevation for each grid point. And to do that, we'll use a DEM. That'll tell us
at any point the elevation. So now with that we can actually say for our point here for this flight the CPA time of 224634 but the delayed CPA time of 224646.
So we do this for every point here and that's one flight done. Then we just do that for all flights in the area and we're left with this grid of points because basically every point heard at least one flight especially since I gave the 10 km radius. But for each of these here, let me zoom in on one. For each of these, what I have is a list of flight IDs and the associated delayed CPA times. Basically, if you were standing here on this day, at each of these times, you could look up and you could hear a plane. So, back to our goal. What
we want is somewhere that heard a plane and then 920 seconds later heard another one. So for each of these points, we then just check for that offset. Here's what that actually looks like. So here's an example point and each of those delayed CPA times. And then we have a 920 second offset window. And we just look here. Nothing. 920 seconds later.
This one again. Not quite. And then this one over here. There we go. Now, when we say 920 seconds apart, it matters where we're measuring from. Like are you calling the center here or here or here? And that's like a plus or minus 5second variance. So that's why if you look closely on this, there's a bit of wiggle room that we're allowing for in that 920 second offset. But anyway, that's the methodology. So how many of our points survived that filter? It's about 35,000.
And that's for this small test area, which is less than 0.05% of the continental US land area that we need to search. Like just a side tangent, I don't think people understand how much land area there is in the US. If you divided the continental US into a bunch of football fields and laid them end to end, it would wrap all the way around the Earth and to the moon and back almost 200 times. That's what we're
searching for this. So, we need to do a lot better and we can, but we need to know some things about Doves. All jokes aside, this is actually so cool. Doppler curves will give us a ton of additional information and I actually don't need to teach you anything because you already know this intuitively. Let me play you two clips.
You can already tell that clip 2 was going faster. The reason is because the drop in frequency was much larger. And that drop in frequency can actually tell us the exact speed that an object was going. So assuming linear constant velocity, there are only two things that determine what this Doppler curve looks like. The velocity determines how big of a frequency drop. And our CPA distance, remember that one, determines how quickly that drop occurs. So for the Doppler curve in our audio, we can fit a
curve to that and know how fast the aircraft was going and how far away it was from the listener. So now instead of filtering to points that hurt an aircraft and then 920 seconds later heard another one, we can filter to points that heard an aircraft going a certain speed at a certain CPA distance and then 920 seconds later heard another plane going a certain speed at a certain CPA distance. That's a lot more constraining. So let's measure one of
the Doppler curves. You can see that the velocity makes the frequency drop larger and then the CPA slant distance just changes how quickly that that drop occurs. What you'll notice on this one is that it doesn't fit a perfect Doppler curve though like this right hand side does. Left hand side not so much.
That'll be important later. But what we do in the meantime is we just take a pretty generous range for velocity so that we don't accidentally filter anything out. So here's kind of a decent fit on the right side at 100 m/s. I can't confidently say this plane was going exactly 100, but we can be pretty sure it wasn't going 70. Similarly for the time offset, there's still a couple seconds of like potential measurement error here. Okay, best match 15115. I'll move it to 15113. Slide that fundamental
frequency up a little bit. And like, yeah, you know, that could work. That's just to say we still can't pin down the offset time precisely to the second. But we do this for event one and event two, and we get the measurement ranges for each. So, the first thing is the speed filters. Here are all of the prop flights, but here are all the ones matching the event one velocity range.
And then in pink, here are the same ones before event two. So now what we're looking for is a point that heard a green line and then 920 seconds later heard a pink line. But remember, the other thing we know about this too, in addition to the velocity, is the distance away. So on this green line for event one, we don't need to lay that entire grid from 0 to 10 km away. We know the pretty precise slant distance that the observer was from each of these. So, here are those points.
They're about 73,000. But if I zoom into any of these clusters, what do you notice? Airport. Another airport. Most of those 73,000 points are just near airports. And it makes sense cuz that's where the planes are. More air traffic just means higher chance that you happen to have two flights that meet those criteria. But the signal in our video is not only what
we hear, it's also what we don't hear. There's no Doppler curve between these two. That's almost 15 minutes of not hearing a plane. You're not going to get that right outside of a major airport. And that's part of why this is so useful cuz if you're near an airport, you hear a lot of flights and you get a lot of signal. If you're not, then the silence itself is also a signal. So it really works anywhere on that spectrum. So, what that means for our search, I'll
take all of these 73,000 points and I'll apply a silence filter. Feel free to pause here if you're interested in the specific thresholds and logic I used, but overall I just disqualified any point that would have hurt a flight between our two events, and I was pretty conservative with the thresholds. So, to recap, we took our velocity filtered flight segments for event one and event two, laid candidate points based on the slant distance range from the Doppler curve measurements. Then for all those