postgis database

[pedromorgan]

postgis is a geospatial database..

We start from there and an new db called cflogs

[pedromorgan]

This is all the stuff gral (miss him) were on about;..
Into mapnick..

and the potgis database and map layers..
its all there in https://github.com/freeflightsim/fg-navdb

[pedromorgan]

That isue to remind @geoffmcl himself is why it was fantastic we actuallt had an
mpserver on fgx.ch and crossfeed purpose (instead of telnet interface)

The next step from there we dicuss was to Aqquire data from the fg=tracker logger wherever that is..

[theo-armour]

@pedromorgan

Visit http://navdb.freeflightsim.org

It says 'authentication required'...

[geoffmcl]

@pedromorgan

That isue to remind @geoffmcl...

And I hope I do not need to remind @pedromorgan that @geoffmcl wrote crossfeed, precisely because he did not want the mpservers telnet interface used - called it telnet bashing ;=)) and is a dev on FGMS (mpserver), so seems very intimate with these components...

Aside from the json/xml feeds, the crossfeed client has quite a number of other features not presently used... some have to be compiled in, if the right dependency libraries exist... and thus have not been well tested...

At present, as explained at length, current GitLab crossfeed can not be re-built in m2, due to missing SimGearCore.a, and maybe other things... all perfectly solvable, but is work to do... if really required...

to Aqquire data from the fg=tracker logger wherever that is..

Well, I for sure do not know whereever that is... in fact even what that is... What exactly are you referring to? If you are referring to the above crossfeed build options, then that has the problems stated...

Alternatively, I started acquiring cf log data, using the json feed... each 5 secs... works great... simple...

So for now some 34 or so days have been acquired of the cflog data... in compact CSV form... only requires you, @pedromorgan, to setup the database, and pass on the specs...

When we have caught up with some 34 days of loaded data, have some http query capabilities, we can discuss how to automatically add ongoing data as it happens... that would be quite easy... but not by re-compiling crossfeed...

Let's get this swinging ;=)) if all is agreed...

[pedromorgan]

@geoffmcl re crossfeed....

That is currently in "/home/fg" user.. If you want to go ahead and compile simgear etc in /home/geoff then please have fun..

Only thing i ask is to make sure that any packages installed dont install "x11" or alike.. ie a headless GUI...

[pedromorgan]

@geoffmcl quick question.. is cfjsonlog.pl in a repository somewhere.. wanna have a look at it.. for curious

[geoffmcl]

@pedromorgan hi Pete,

Re: crossfeed

Yes, that the current running crossfeed is currently in /home/fg/crossfeed, but that is from the now dead gitorious.org... that runs fine, and has done so, for many years... it is the current source of -
http://crossfeed.freeflightsim.org/flights.json - it works fine - no problems... no change required here...

To upgrade this to the most recent GitLab crossfeed source requires SG core library, at least... Now I do not fully understand your restriction as to no X11 install, but I also would not need that, so ok... and maybe this is not required... to build OSG... but OSG is required by SG...

But really crossfeed does not require X11... for sure, and crossfeed only links the the SimGearCore.a library, which should have no video, aka x11 requirement... so we should be good... will try a geoff a download_and_compile.sh run and see what happens...

Oops, seems a cmake upgrade required...

CMake Error at CMakeLists.txt:1 (cmake_minimum_required):
  CMake 2.8.11 or higher is required.  You are running version 2.8.9

Just a few points different... How do do this?

is cfjsonlog.pl in a repository somewhere

Yes, of course... cfjsonlog.pl... check it out... is a very quiet 5 second reader of http, of cf json... it has some filters, but essentially writes each crossfeed json fetch to a CSV daily file name...

The 5 second fetch, and the filters, are important to a crossfeed tracker data accumulation... to me, for tracking a flight, each 5 secs is very sufficient... more has no meaning... and the filters are if the flight continues for more that 60 seconds, and/or moves more than 2 km (or 15 mins), so this helps reduce really not needed data bloat... a very concise daily CSV file...

I did think about the output being in json, especially since the information received is in json, but chose a more concise CSV format... as I read somewhere, to write it in json would more than double the current 30-40 MB daily file size...

So the output it in concise CSV format... easy to read by anyone... seems no need for the json bloat here?

BUT the best part is when I read those CSV daily logs with cgcsvlogs.pl, recently adjust to use only UTC time, and output something else...

Right now it outputs some html, like 20160814, but as stated, once I know the pg database specs, it could easily out an update SQL statement... even run it if you want the script to go that far...

Now I have looked at the old crossfeed tracker database, creat_pg.sql, and it seems this could be used again here...

While the CSV offers two more fields to the position table, maybe they are not really neccessay?

But cfcsvlogs.pl could be easily coached to output SQL add records, thus populate the intital cflogs database...

So where next? Need cmake update to continue SG creation... Need discussion on the flights, and position tables, in a cflogs database...

Seems to be moving forward ;=))

OT: Just ran m2 /home/geoff/bin/updcflog.sh, and added flights-2016-08-17.csv to the acquired data... seems working good... in a few days, will add it to a geoff cron each morning...

[theo-armour]

@fgx/owners

I am unclear about things.

Are there CSV replay files now available somewhere on freeFlightSim.org - or not yet?

If so, what is the link?

[geoffmcl]

@theo-armour no URL link to freeflightsim yet... the first step is to create cflog database... WIP...

Of course the CSV files are now being created on m2... the server powering freeflightsim... and pushed to this github repo... and my previous comment should have read flights-2016-08-16.csv... of course the 17th, today, is in the process of being created, growing every 5 seconds...

As I understand it... the specs are being written to put all that CSV data into a PostgesSQL database... WIP...

Then I imagine eventually something like say cflog.freeflightsim.org/?callsign=THEO&form=json would return all your flights, your FIDs, in json, both closed - ie previous, historic flights, and, eventually, open - ie what you are flying now... each with a positions array... if you have ANY! Just the result of an sql query...

When that link, or its like, is active, you could write a flights.html, that got the json/csv list for just one flight, and displayed the track over a 3D terrain... like you are doing now, reading the cooked CSV files... immediately many more tracks to be show...

And if you want to show one days flights, of all FIDs, exactly like you are doing now on the globe, maybe say cflog.ffs?date=2016-08-16&form=csv would essentially re-create that daily, now from the database...

Of course this is no gain, since that is what you have now reading these CSV dailies, but I hope you can see what other possibilities this opens... but all WIP...

So no links yet... but hopefully, one day, maybe... I am sure Pete will correct me if I am running off the rails here ;=))

Regards,
Geoff.

[pedromorgan]

So fun is starting here..
https://github.com/freeflightsim/crossfeed-logger

So note is the "router", which dictates where the end points will be..
https://github.com/freeflightsim/crossfeed-logger/blob/master/server/init_run.go#L19

As @geoffmcl mentioned, the idea is to have a database with tables of callsigns, flights, aircraft used, path and timings...

Important to note is that the "flight" is made up of postiions, ie lat,lon,alt,time and will be stored in the database as a single row, where the positions are a GeoLine .

http://postgis.net/docs/ST_MakeLine.html

[pedromorgan]

@geoffmcl whats the phrase u got over the stream in paris.. deja vue .. Guess what i found..https://github.com/fgx/flightgear-tracker-go

And thats when we were chatting to the guy in honk-kong who does the tracker...

[pedromorgan]

@geoffmcl Note the GetFlight and GetFlightTrack
https://github.com/fgx/flightgear-tracker-go/blob/master/tracker/flight.go#L79