OpenAerialMap/Meeting Feb 05, 2015

IRC Log

Feb 05 18:01:34 Cristiano: Good morning everyone!
Feb 05 18:02:24 Fraank: Morning
Feb 05 18:02:42 BlakeGirardot: Hi
Feb 05 18:02:58 Cristiano: We are doing the OAM weekly 1 hour early to not overlap with the HOT Summit planning at 18UTC
Feb 05 18:03:44 Cristiano: Let's wait a couple more minutes to see if anyone else shows up. Hi Fraank and Blake!
Feb 05 18:04:35 Cristiano: Blake: are you doing the session log as the previous times?
Feb 05 18:04:44 BlakeGirardot: Yes
Feb 05 18:05:04 BlakeGirardot: Is the previous survey link still the right one for review?
Feb 05 18:05:29 BlakeGirardot: https://docs.google.com/forms/d/13tqfTHbSgw-yVZMdVmiGkYRAxJiILqHvP5-tORizhJY/viewform?c=0&w=1
Feb 05 18:05:34 BlakeGirardot: That one?
Feb 05 18:05:43 Cristiano: Yes, it should be... shorter link: [rejected by wiki software] Feb 05 18:06:00 Cristiano: It's now split in three sections
Feb 05 18:06:17 BlakeGirardot: I see.
Feb 05 18:06:24 BlakeGirardot: I mean: Oh, I see that now.
Feb 05 18:07:16 Cristiano: Here's the link to the agenda: https://docs.google.com/document/d/1bxv6gO4qWJyZ-J6BN2KZfS979O1UPFyg9oBUiHAtPCo/edit?usp=sharing
Feb 05 18:08:07 Cristiano: If there are no further comments I'd like to start sending out the link
Feb 05 18:09:00 BlakeGirardot: (I'm reading it over one more time)
Feb 05 18:09:18 Fraank: looks good to me
Feb 05 18:10:32 BlakeGirardot: Me too
Feb 05 18:11:14 BlakeGirardot: Unfortunately, I have to step out for about 15 mins to go pick up my wife at the train station (didn't know she was coming home late today).
Feb 05 18:11:23 BlakeGirardot: Be back as soon as possible.
Feb 05 18:11:27 Cristiano: Good. Feel free to "snow-ball" it to your contacts
Feb 05 18:11:35 Cristiano: no worries
Feb 05 18:12:27 Cristiano: I will also send an email out to the list explaining the context of the OAM project and the purpose of the survey
Feb 05 18:14:01 Cristiano: Let's move onto the catalog design. Any comment/suggestion?
Feb 05 18:14:41 Cristiano: I put a link to the rough draft on GitHub: https://github.com/hotosm/OpenAerialMap/issues/5
Feb 05 18:14:52 Fraank: looking over it now
Feb 05 18:16:40 Cristiano: and while you are there, what technology would you recommend for each component?
Feb 05 18:17:29 Cristiano: keeping in mind the general requirements and the examples I posted in the agenda
Feb 05 18:17:37 Fraank: right
Feb 05 18:20:22 Fraank: as far as ensuring the system is distributed, does that limit any of the web interfaces?
Feb 05 18:21:22 Cristiano: In what sense?
Feb 05 18:23:02 Fraank: sorry, I guess could you clarify by what is meant by the 'high availability' requirement
Feb 05 18:25:01 Cristiano: oh, that probably applies more to the actual tile services, but it can also refer to the main OAM Website in general, so that it scales access during peak times
Feb 05 18:25:24 Fraank: ok
Feb 05 18:26:13 Cristiano: let's say after a disaster a lot of people go to oam.hotosm.org to look for what imagery is available, we want to make sure it stays available
Feb 05 18:26:58 Cristiano: and then of course not just the Web interface, but also the services once people start pointing to the same TMS
Feb 05 18:27:12 Fraank: sure
Feb 05 18:28:18 Cristiano: So, anyway from the other discussion about the database choice, it sounds like we should stick with something like SQLite or MySQL
Feb 05 18:28:21 BlakeGirardot: I thought there was a little more 2 way communications between local machines and remote nodes
Feb 05 18:29:05 BlakeGirardot: Or local machines and central node I guess
Feb 05 18:29:08 Cristiano: I am no expert with NoSQL options, but from what I was reading since we are not dealing with very complex data, they may be easier for scaling and distributed systems
Feb 05 18:31:12 Cristiano: I was thinking that if the index is replicated everywhere, then each node just has to tell the OAM network if it's alive and what capacity it has...
Feb 05 18:32:09 BlakeGirardot: Somewhere in the existing documents I thought a local node could communicate with the central index to get catalog info and pull imagery, then disconnect and serve imagery locally
Feb 05 18:32:17 Cristiano: Basically each node can have a full OAM index/catalog and services or just serving TMS services
Feb 05 18:32:53 BlakeGirardot: http://oam.osgeo.org
Feb 05 18:32:57 BlakeGirardot: oops wrong window
Feb 05 18:33:19 Cristiano: Yes, that could be a good feature to implement to load balance
Feb 05 18:34:27 BlakeGirardot: Anyway, so I though the local to the remote nodes, one of which could be a "central" node, should have arrows on both ends
Feb 05 18:35:41 BlakeGirardot: or Local Database doesn't directly communicate with remote nodes
Feb 05 18:36:03 BlakeGirardot: Local OAM server would have bi-directional arrow to the remote nodes
Feb 05 18:36:14 Cristiano: https://docs.google.com/drawings/d/1ayU0tS5lIE0JoIuB3dmuOe21golaz85gN576PrvVmNo/edit?usp=sharing
Feb 05 18:36:16 BlakeGirardot: and there would be no local db to remote nodes connection?
Feb 05 18:36:48 BlakeGirardot: That is the document I am referring to
Feb 05 18:37:09 Cristiano: Please make your versions of it and propose it on the GitHub issue discussion ;-)
Feb 05 18:37:19 BlakeGirardot: Oh, ok, can do
Feb 05 18:38:52 Cristiano: Yes, I guess the nodes in the current drawing only act as tile servers, but they could be full OAM nodes instead, with index, catalog and everything
Feb 05 18:39:30 BlakeGirardot: Oh I see, yes, that could be it, terminology. I was thinking a "whole package" db, server, etc was a "node"
Feb 05 18:40:33 BlakeGirardot: I'll mess around with the copy I made and work some of dodobas's terminology in there
Feb 05 18:40:50 BlakeGirardot: http://dodobas.github.io/OpenAerialMap/oam-components/
Feb 05 18:41:04 BlakeGirardot: And see if that matches what you are envisioning.
Feb 05 18:42:14 Cristiano: Actually, I think we need to simplify from that original design, otherwise it may be hard to make it portable
Feb 05 18:43:15 BlakeGirardot: I agree
Feb 05 18:43:17 Cristiano: Or at least know that in portable versions all OAM functions will sit in the same container
Feb 05 18:45:02 BlakeGirardot: As to the database, I tend to agree with Mark Cupitt, Postgresql with postgis makes most sense to me.
Feb 05 18:45:46 Cristiano: Do we need the complexity of Postgres and Postgis?
Feb 05 18:46:41 Cristiano: I mean, they are excellent options, but we're only really storying basic metadata and simple BBOXes in GeoJSON...
Feb 05 18:47:08 BlakeGirardot: Is that all we need, we won't need more spatially aware db for anything?
Feb 05 18:47:36 BlakeGirardot: We need postgres and postgis just for the tasking manager now
Feb 05 18:47:52 BlakeGirardot: I think because the bboxes are not always boxes maybe?
Feb 05 18:48:14 Cristiano: And how easy/fast is to deploy those options into something portable in nix/win/mac/etc?
Feb 05 18:48:26 BlakeGirardot: This one would be good to get some input from dodobas on
Feb 05 18:48:34 Cristiano: yes, I think the TM has more spatial requirements
Feb 05 18:48:46 BlakeGirardot: pretty easy to spin up postgres/postgis
Feb 05 18:49:13 Cristiano: yes, I was hoping he would made the meeting...
Feb 05 18:49:15 BlakeGirardot: oh wait
Feb 05 18:49:21 BlakeGirardot: win
Feb 05 18:49:26 BlakeGirardot: not sure about that
Feb 05 18:50:26 Cristiano: right... I'm talking about a click/click setup for someone who goes out to a disaster area, flies a UAV and then wants to share that imagery with his local team
Feb 05 18:51:03 BlakeGirardot: Ya, I would like that too
Feb 05 18:51:10 Cristiano: OK, anyway, let's discuss options and continue the discussion about the catalog architechture on GitHub directly. Related to that, we're trying to hire pretty soon to start working on the implementation
Feb 05 18:51:33 Cristiano: so that brings me to the next point on the agenda :-)
Feb 05 18:51:49 Cristiano: (and trying to finish by 10)
Feb 05 18:52:01 Cristiano: (pacific time sorry)
Feb 05 18:52:57 Cristiano: I started drafting the tech challenge, but I would like your input and comments
Feb 05 18:53:30 BlakeGirardot: (im reading linked doc)
Feb 05 18:54:09 Cristiano: It's linked in the agenda. One of the main requirements is that who is going to do the job needs to have a real commitment to make it sustainable
Feb 05 18:54:29 BlakeGirardot: Well the pay seems good :)
Feb 05 18:54:51 Cristiano: He would be leading the development and coordinate with the community contributors
Feb 05 18:54:53 BlakeGirardot: Meaning that should get a dedicated person and hopefully
Feb 05 18:55:15 Cristiano: Pay is another thing that I'd like to get your input on :-)
Feb 05 18:55:52 Cristiano: there's over $100K in total budget
Feb 05 18:56:11 BlakeGirardot: Those figures are for 6 months of full time work?
Feb 05 18:56:15 Cristiano: I mean, not just for the catalog, but for the full OAM implementation
Feb 05 18:56:47 Cristiano: Well, 6 months plus some maintanence after
Feb 05 18:57:06 Cristiano: or if you guys have any other ideas to ensure code sustainability
Feb 05 18:57:06 BlakeGirardot: 6 + 6 about?
Feb 05 18:57:28 Cristiano: yes, something like 10-20 hours per month for the other 6 months
Feb 05 18:57:40 Cristiano: to cover some bug fixing etc
Feb 05 18:58:18 BlakeGirardot: Ya, that is kind of what I was thinking too
Feb 05 18:59:03 Cristiano: But, we can also do two part time developers, or adjust the budget based on what we think is the catalog effort vs the other components
Feb 05 19:00:14 Cristiano: So, I even though of crowdsourcing the entire challenge creation, to fully involve potential suppliers (see the other link), but that may take longer, and we need to get someone started soon
Feb 05 19:00:58 BlakeGirardot: I see. I need to read the other linked docs
Feb 05 19:01:10 BlakeGirardot: Some email list follow ups on this before next week
Feb 05 19:01:27 Cristiano: Anyway, I'd like to get your comments and ideas on that, it looks like our time is up
Feb 05 19:01:47 Cristiano: yes, I will send an email for comments to the list
Feb 05 19:02:12 Cristiano: Unless there anything else, let's wrap up and jump onto the HOT summit meeting
Feb 05 19:02:18 BlakeGirardot: ok, cool. I have a few questions related to your ideas for the dev process and the tech challeng/crowd source parts
Feb 05 19:02:31 BlakeGirardot: Ya, lets wrap
Feb 05 19:02:47 Cristiano: Cool - thanks guys! Let's talk in a week
Feb 05 19:02:50 BlakeGirardot: and we can talk email or otherwise next week :)
Feb 05 19:02:54 BlakeGirardot: Thank you Cristiano !
Feb 05 19:02:58 BlakeGirardot: And Fraank
Feb 05 19:02:59 Cristiano: I mean talk to you in a couple of minute Blake :-)
Feb 05 19:03:02 BlakeGirardot: :)
Feb 05 19:50:36 dodobas: Cristiano: sqlite is available everywhere ... easy to depoly
Feb 05 19:51:10 dodobas: and we don't really need anything spatial... but there is also spatialite ...built on top of sqlite ...
Feb 05 19:52:37 dodobas: and you could push sqlite even further... for example, use it for 'packaging' and package tiles and layers in a single file
Feb 05 19:53:01 dodobas: which would be easy to 'move' aroun