How to create baseline chart using ADORE
NOTE: After publishing this tutorial I heard that supersites website is not supposed to have SLC
images stored and these images were there due to a mistake. You might not be able to download them
in the future.
In this tutorial we will use the Envisat SLC images over Istanbul, made available by the Supersites
project (http://supersites.earthobservations.org/). If you would like to follow the tutorial step by step,
you will need to download the data from Supersites, which requires you apply for access if you do not
already have one. The list of files (Istanbul_ASA_IMS_T429.txt) is given at the end of this tutorial as
an appendix. If you have the Supersites login you can download the data with the following commands:
mkdir -p ~/Downloads/Istanbul
cd ~/Downloads/Istanbul
wget -nd --user="yourSupersitesUsername" --password="yourSupersitesPassword" -i
~/Desktop/Istanbul_ASA_IMS_T429.txt
We start with creating a new project folder. I will use the ~/playground/Istanbul folder on my computer.
This folder can be anywhere on your computer, no need to use the same location.
mkdir -p ~/playground/Istanbul
cd ~/playground/Istanbul
Then we start ADORE as usual.
adore
-i
The first thing we will do is to create a data folder with the Envisat files in it. There is a new
functionality added after r134. Using this function we can create the links to the data files and create
the right structure for ADORE.
scenes init ~/Downloads/Istanbul
Now we can create a settings file using:
ADORE: settings init
USAGE:
. settings apply variableName
ex: . settings apply master
What is the projectFolder? [.]
/home/bosmanoglu/playground/Istanbul
projectFolder="/home/bosmanoglu/playground/Istanbul"
adoreHistoryFile=/home/bosmanoglu/playground/Istanbul/.history
dataFolder=/home/bosmanoglu/playground/Istanbul/data
processFolder=/home/bosmanoglu/playground/Istanbul/process
tmpFolder=/home/bosmanoglu/playground/Istanbul/tmp
Applying all suggested changes.
m_crop_in="dummy"
runFolder="/home/bosmanoglu/playground/Istanbul/process/default"
s_crop_in="dummy"
What is the runName? [default]
highResDem
runName="highResDem"
logfile=./highResDem.log
runFolder=/home/bosmanoglu/playground/Istanbul/process/highResDem
Applying all suggested changes.
baselinesFolder="/home/bosmanoglu/playground/Istanbul/process/highResDem/baselines
"
cropsFolder="/home/bosmanoglu/playground/Istanbul/process/highResDem/crops"
excludesFile="/home/bosmanoglu/playground/Istanbul/process/highResDem/excludes"
i12sFolder="/home/bosmanoglu/playground/Istanbul/process/highResDem/i12s"
What is the dataFolder? [/home/bosmanoglu/playground/Istanbul/data]
In the dataFolder I found the following scenes:
12006 12507 13008 14010 15012 17016 18018 21024 22527 23028 24531 26034 27036
30042 32547 33048 33549 34551 35052 37056 37557 38058 38559 39060 40062 4992 5994
7497 8499 9000
Do you want to exclude any scenes? [y/n] n
What is the dataFile pattern? [DAT_01.001]
Here is the listing of the files in the current master folder.
ASA_IMS_1PNUPA20040616_194716_000000162027_00429_12006_5633.N1
*.N1
dataFile="*.N1"
m_in_dat=/home/bosmanoglu/playground/Istanbul/data/12006/ASA_IMS_1PNUPA20040616_19
4716_000000162027_00429_12006_5633.N1
s_in_dat=/home/bosmanoglu/playground/Istanbul/data/12507/ASA_IMS_1PNUPA20040721_19
4716_000000152028_00429_12507_4524.N1
Applying all suggested changes.
m_crop_in="/home/bosmanoglu/playground/Istanbul/data/12006/ASA_IMS_1PNUPA20040616_
194716_000000162027_00429_12006_5633.N1"
s_crop_in="/home/bosmanoglu/playground/Istanbul/data/12507/ASA_IMS_1PNUPA20040721_
194716_000000152028_00429_12507_4524.N1"
Do you also want to enter the patterns for leader, volume and null files? [y/n] n
What is the m_in_method? Available options are: ERS, ASAR, RSAT, ATLANTIS, JERS,
ALOS, TSX, CSK, GAMMA
Current setting: [ERS]
ASAR
m_in_method="ASAR"
s_in_method=ASAR
Applying all suggested changes.
What is the m_orbdir? This is the folder that has the arclist file for getorb.
Current setting: []
Applying all suggested changes.
What would you like to name this settings file?
Current setting: [settings.set]
Please wait while I check the current environment...
Settings saved to settings.set
ADORE:
Note that we did not specify a folder for the precise orbits step. In this case we will use the orbits
specified in the SLC image files.
Before we start the baselines generation script, if you are using a new installation of ADORE you may
want to check that the setup is correct using:
ADORE: check setup
ADORE scripts are in: /opt/adore/scr
csh in /bin/csh
doris in /usr/local/bin/doris
cpxfiddle in /usr/local/bin/cpxfiddle
envisat_dump_data in /usr/local/bin/envisat_dump_data
psxy in /usr/lib/gmt/bin/psxy
gnuplot in /usr/bin/gnuplot
convert in /usr/bin/convert
gdal_translate in /usr/bin/gdal_translate
If there were to be any missing binary files (executables that ADORE call) we would get a warning
message.
There are a few template scripts stored in the $ADOREFOLDER/templates folder.
ADORE: ls $ADOREFOLDER/templates
baselines.adr
preparePsiInput.adr
singleMasterStack_Parallel.adr
multiplePairs_Parallel.adr singleMasterStack.adr template.drs
The names are pretty self explanatory, and we will be using the baselines.adr script. We can go ahead
and run this script directly as:
p $ADOREFOLDER/templates/baselines.adr
If any modifications are needed to the scripts it is advised to make a new copy of the script and make
changes on the new copy. For the baselines script we don't need to make any changes. The
precise_orbits step will fail, because we did not provide the m_orbdir location. But this does not affect
the script to plot the baselines using the orbits given in the SLC image.
When the script finishes, we get the following messages:
Normal termination.
Thank you for using Doris.
coarseorb: SUCCESS
minmax: Mismatch between actual (1) and expected (3) fields near line 29
minmax: Mismatch between actual (1) and expected (3) fields near line 29
Temporal-Perpendicular baseline plot:
/home/bosmanoglu/playground/Istanbul/process/highResDem/baselines/baselines.eps
The error message (related to minmax) indicates there was a problem with the baseline generation. This
is likely due to bad/corrupt orbits in one file. We can check this by looking at the reported baselines
using:
ADORE: cat
/home/bosmanoglu/playground/Istanbul/process/highResDem/baselines/baselines
0
0
12006
35
196.7
12507
70
779.8
13008
140
37
14010
210
-42.7
15012
350
519.1
17016
420
1308.9
18018
630
1041.5
21024
735
-202.9
22527
875
850.5
23028
980
216.2
24531
1050 53.8
26034
1260
1435
1470
1505
1575
1610
1750
1785
1820
1855
1890
1960
-490
-420
-315
-245
-210
897.5
280.4
414.9
264.4
165.1
692.1
797.1
43.4
109.1
534
455.4
250
1219.9
-384.2
277.9
571.8
494
27036
30042
32547
33048
33549
34551
35052
37056
37557
38058
38559
39060
40062
4992
5994
7497
8499
9000
As you can see the last orbit is missing its baseline values. However we do not know which scene had
the problem. In order to find that out we can check the file size of the result files to see if one of them is
different:
ADORE: cd $runFolder/baselines
ADORE: ls -lh *_*.res
-rw-r--r-- 1 bosmanoglu bosmanoglu
-rw-r--r-- 1 bosmanoglu bosmanoglu
-rw-r--r-- 1 bosmanoglu bosmanoglu
-rw-r--r-- 1 bosmanoglu bosmanoglu
-rw-r--r-- 1 bosmanoglu bosmanoglu
-rw-r--r-- 1 bosmanoglu bosmanoglu
-rw-r--r-- 1 bosmanoglu bosmanoglu
-rw-r--r-- 1 bosmanoglu bosmanoglu
-rw-r--r-- 1 bosmanoglu bosmanoglu
-rw-r--r-- 1 bosmanoglu bosmanoglu
-rw-r--r-- 1 bosmanoglu bosmanoglu
-rw-r--r-- 1 bosmanoglu bosmanoglu
-rw-r--r-- 1 bosmanoglu bosmanoglu
-rw-r--r-- 1 bosmanoglu bosmanoglu
-rw-r--r-- 1 bosmanoglu bosmanoglu
-rw-r--r-- 1 bosmanoglu bosmanoglu
-rw-r--r-- 1 bosmanoglu bosmanoglu
-rw-r--r-- 1 bosmanoglu bosmanoglu
-rw-r--r-- 1 bosmanoglu bosmanoglu
-rw-r--r-- 1 bosmanoglu bosmanoglu
-rw-r--r-- 1 bosmanoglu bosmanoglu
-rw-r--r-- 1 bosmanoglu bosmanoglu
-rw-r--r-- 1 bosmanoglu bosmanoglu
-rw-r--r-- 1 bosmanoglu bosmanoglu
-rw-r--r-- 1 bosmanoglu bosmanoglu
-rw-r--r-- 1 bosmanoglu bosmanoglu
-rw-r--r-- 1 bosmanoglu bosmanoglu
-rw-r--r-- 1 bosmanoglu bosmanoglu
-rw-r--r-- 1 bosmanoglu bosmanoglu
2.8K
2.8K
2.8K
2.8K
2.8K
2.8K
2.8K
2.8K
1.3K
2.8K
2.8K
2.8K
2.8K
2.8K
2.8K
2.8K
2.8K
2.8K
2.8K
2.8K
2.8K
2.8K
2.8K
2.8K
2.8K
2.8K
2.8K
2.8K
2.8K
2011-02-18
2011-02-18
2011-02-18
2011-02-18
2011-02-18
2011-02-18
2011-02-18
2011-02-18
2011-02-18
2011-02-18
2011-02-18
2011-02-18
2011-02-18
2011-02-18
2011-02-18
2011-02-18
2011-02-18
2011-02-18
2011-02-18
2011-02-18
2011-02-18
2011-02-18
2011-02-18
2011-02-18
2011-02-18
2011-02-18
2011-02-18
2011-02-18
2011-02-18
10:11
10:11
10:11
10:11
10:11
10:11
10:11
10:11
10:11
10:11
10:11
10:11
10:11
10:11
10:11
10:12
10:12
10:12
10:12
10:12
10:12
10:12
10:12
10:12
10:12
10:12
10:12
10:12
10:12
12006_12507.res
12006_13008.res
12006_14010.res
12006_15012.res
12006_17016.res
12006_18018.res
12006_21024.res
12006_22527.res
12006_23028.res
12006_24531.res
12006_26034.res
12006_27036.res
12006_30042.res
12006_32547.res
12006_33048.res
12006_33549.res
12006_34551.res
12006_35052.res
12006_37056.res
12006_37557.res
12006_38058.res
12006_38559.res
12006_39060.res
12006_40062.res
12006_4992.res
12006_5994.res
12006_7497.res
12006_8499.res
12006_9000.res
As you can see the highlighted file is about half the size as the others, and is probably the file we are
looking for. Looking at the result file for 23028 we see the following:
ADORE: less 23028.res
*******************************************************************
*_Start_leader_datapoints
*******************************************************************
t(s)
X(m)
Y(m)
Z(m)
NUMBER_OF_DATAPOINTS:
0
*******************************************************************
* End_leader_datapoints:_NORMAL
*******************************************************************
#Report that else next line could easily be too long:
Datafile:
/home/bosmanoglu/playground/Istanbul/data/23028/ASA_IMS_1PNUPA20060726_194703_0000
00162049_00429_23028_5739.N1/home/bosmanoglu/playground/Istanbul/data/23028/ASA_IM
S_1PNUPA20060726_194715_000000162049_00429_23028_5722.N1
Dataformat: ENVISAT
Number_of_lines_original:
Number_of_pixels_original:
*******************************************************************
* End_readfiles:_NORMAL
*******************************************************************
As you can see there is a problem with the datafile. We can check the dataFolder to see what the
problem is:
ADORE: ls $dataFolder/23028
ASA_IMS_1PNUPA20060726_194703_000000162049_00429_23028_5739.N1
ASA_IMS_1PNUPA20060726_194715_000000162049_00429_23028_5722.N1
It seems like there were two files with the same orbit number in the L1 folder, and that caused
problems. ADORE expects one scene per orbit folder. I know that the one ending with 5722.N1 is the
file that we need. I'll delete the other file and re-run the baselines calculation.
ADORE: rm -rf
$dataFolder/23028/ASA_IMS_1PNUPA20060726_194703_000000162049_00429_23028_5739.N1
ADORE: cd $projectFolder
I will also go ahead and remove the old baselines directory, since we will be creating a new one. If you
want to keep the old one, you can change the runName using "settings apply -r
runName=highResDem2".
ADORE: rm -rf $runFolder/baselines
ADORE: p ${ADOREFOLDER}/templates/baselines.adr
So this time we did not get an error message from minmax, and we can take a look at the file at its
reported location:
Temporal-Perpendicular baseline plot:
/home/bosmanoglu/playground/Istanbul/process/highResDem/baselines/baselines.eps
We see something similar to this:
In the above plot, the temporal baselines are given in the "x-axis" and the perpendicular baselines are
given in the y-axis. The scattered numbers are the "folder names" in the data folder. Since we used the
orbit numbers in the data folder, this is what we see here. The baselines are given according to the orbit
12006, which was selected automatically. The script selects the first orbit as master, and 12006 is the
first orbit when sorted alphabetically. The labels are plotted such that their center coincide with the
values in the text file, or in other words, the connecting lines in the above plot, goes through the center
of the labels.
If you like to convert the EPS file to another raster format you can use the convert command. For
example to convert the EPS to PNG (type all in one line):
convert
/home/bosmanoglu/playground/Istanbul/process/highResDem/baselines/baselines.eps
/home/bosmanoglu/playground/Istanbul/process/highResDem/baselines/baselines.png
Using Different Orbits
Previously we looked at the baselines calculated using the orbits stored in the header of the image file.
We can also look at the baselines calculated using precise orbits. For this case, I will use the DORVOR orbits from ESA. For more information see: http://www.osmanoglu.org/index.php/sar/6sarprocessing/58-dorvortoodrconversion
I have these orbits _converted_ and stored at the following location:
/home/bosmanoglu/data/orbits/DOR.ODR/
I would also like to keep the previous calculation, so I will change the runName as well.
ADORE: cd $projectFolder
ADORE: settings apply -r runName=highResDemDorVor
m_orbdir="/home/bosmanoglu/data/orbits/DOR.ODR/"
runName="highResDemDorVor"
logfile=./highResDemDorVor.log
runFolder=/home/bosmanoglu/playground/Istanbul/process/highResDemDorVor
m_orbdir="/home/bosmanoglu/data/orbits/DOR.ODR/"
s_orbdir=/home/bosmanoglu/data/orbits/DOR.ODR/
Applying all suggested changes.
baselinesFolder="/home/bosmanoglu/playground/Istanbul/process/highResDemDorVor/bas
elines"
cropsFolder="/home/bosmanoglu/playground/Istanbul/process/highResDemDorVor/crops"
excludesFile="/home/bosmanoglu/playground/Istanbul/process/highResDemDorVor/exclud
es"
i12sFolder="/home/bosmanoglu/playground/Istanbul/process/highResDemDorVor/i12s"
So now we can run the baseline calculations again:
ADORE: p ${ADOREFOLDER}/templates/baselines.adr
...
Many messages...
...
coarseorb: SUCCESS
Temporal-Perpendicular baseline plot:
/home/bosmanoglu/playground/Istanbul/process/highResDemDorVor/baselines/baselines.
eps
And I get something like the following:
As you can see the orbit 4992 is causing trouble. Therefore if we are going to continue processing this
data with the DOR-VOR orbits, we need to exclude this scene from our dataset. We can of course replot the data after deleting the line for orbit 4992, but I would like to show you how to exclude scenes
instead.
ADORE: scenes include
12006 12507 13008 14010 15012 17016 18018 21024 22527 23028 24531 26034 27036
30042 32547 33048 33549 34551 35052 37056 37557 38058 38559 39060 40062 4992 5994
7497 8499 9000
ADORE: scenes exclude 4992
Excluding 4992
And I'll do the same as before, remove baselines folder, and re-run the script.
ADORE: echo $runFolder
/home/bosmanoglu/playground/Istanbul/process/highResDemDorVor
ADORE: rm -rf $runFolder
ADORE: p ${ADOREFOLDER}/templates/baselines.adr
This time we get the following plot:
As you can see there are not much differences between this final plot and the
first result we obtained from the orbits in the header of the image files, except
the fact that 4992 is missing in this plot.
Appendix A - List of files to download
http://eo-virtual-archive2.esa.int/Turkey/ASA_IMS_1PNDPA20061108_194712_000000162052_00429_24531_0435.N1
http://eo-virtual-archive2.esa.int/Turkey/ASA_IMS_1PNDPA20070502_194708_000000162057_00429_27036_0444.N1
http://eo-virtual-archive2.esa.int/Turkey/ASA_IMS_1PNDPA20080521_194703_000000162068_00429_32547_0443.N1
http://eo-virtual-archive2.esa.int/Turkey/ASA_IMS_1PNDPA20091028_194656_000000162083_00429_40062_0447.N1
http://eo-virtual-archive2.esa.int/Turkey/ASA_IMS_1PNUPA20030212_194701_000000162013_00429_04992_4173.N1
http://eo-virtual-archive2.esa.int/Turkey/ASA_IMS_1PNUPA20030423_194708_000000162015_00429_05994_4476.N1
http://eo-virtual-archive2.esa.int/Turkey/ASA_IMS_1PNUPA20030806_194714_000000162018_00429_07497_4488.N1
http://eo-virtual-archive2.esa.int/Turkey/ASA_IMS_1PNUPA20031015_194712_000000162020_00429_08499_4494.N1
http://eo-virtual-archive2.esa.int/Turkey/ASA_IMS_1PNUPA20031119_194710_000000162021_00429_09000_4501.N1
http://eo-virtual-archive2.esa.int/Turkey/ASA_IMS_1PNUPA20040616_194716_000000162027_00429_12006_5633.N1
http://eo-virtual-archive2.esa.int/Turkey/ASA_IMS_1PNUPA20040721_194716_000000152028_00429_12507_4524.N1
http://eo-virtual-archive2.esa.int/Turkey/ASA_IMS_1PNUPA20040825_194714_000000162029_00429_13008_4513.N1
http://eo-virtual-archive2.esa.int/Turkey/ASA_IMS_1PNUPA20041103_194714_000000162031_00429_14010_4519.N1
http://eo-virtual-archive2.esa.int/Turkey/ASA_IMS_1PNUPA20050112_194709_000000162033_00429_15012_5741.N1
http://eo-virtual-archive2.esa.int/Turkey/ASA_IMS_1PNUPA20050601_194716_000000162037_00429_17016_4512.N1
http://eo-virtual-archive2.esa.int/Turkey/ASA_IMS_1PNUPA20050810_194713_000000162039_00429_18018_4530.N1
http://eo-virtual-archive2.esa.int/Turkey/ASA_IMS_1PNUPA20060308_194702_000000162045_00429_21024_4536.N1
http://eo-virtual-archive2.esa.int/Turkey/ASA_IMS_1PNUPA20060621_194712_000000162048_00429_22527_4526.N1
http://eo-virtual-archive2.esa.int/Turkey/ASA_IMS_1PNUPA20060726_194703_000000162049_00429_23028_5739.N1
http://eo-virtual-archive2.esa.int/Turkey/ASA_IMS_1PNUPA20060726_194715_000000162049_00429_23028_5722.N1
http://eo-virtual-archive2.esa.int/Turkey/ASA_IMS_1PNUPA20070221_194707_000000162055_00429_26034_4657.N1
http://eo-virtual-archive2.esa.int/Turkey/ASA_IMS_1PNUPA20071128_194700_000000162063_00429_30042_4663.N1
http://eo-virtual-archive2.esa.int/Turkey/ASA_IMS_1PNUPA20080625_194704_000000162069_00429_33048_4653.N1
http://eo-virtual-archive2.esa.int/Turkey/ASA_IMS_1PNUPA20080730_194704_000000162070_00429_33549_5644.N1
http://eo-virtual-archive2.esa.int/Turkey/ASA_IMS_1PNUPA20081008_194702_000000162072_00429_34551_5650.N1
http://eo-virtual-archive2.esa.int/Turkey/ASA_IMS_1PNUPA20081112_194659_000000162073_00429_35052_5649.N1
http://eo-virtual-archive2.esa.int/Turkey/ASA_IMS_1PNUPA20090401_194658_000000162077_00429_37056_5653.N1
http://eo-virtual-archive2.esa.int/Turkey/ASA_IMS_1PNUPA20090506_194701_000000162078_00429_37557_5657.N1
http://eo-virtual-archive2.esa.int/Turkey/ASA_IMS_1PNUPA20090610_194703_000000162079_00429_38058_5661.N1
http://eo-virtual-archive2.esa.int/Turkey/ASA_IMS_1PNUPA20090715_194702_000000162080_00429_38559_5670.N1
http://eo-virtual-archive2.esa.int/Turkey/ASA_IMS_1PNUPA20090819_194702_000000162081_00429_39060_5669.N1
Copy the above list of files to a text file and save it as ~/Desktop/Istanbul_ASA_IMS_T429.txt