Meeting on April 5, 2010

Agenda

1. Attendance
2. Announce agenda
3. Approve March 22, 2010 Meeting Notes. Ask for review and address any problems seen in the notes.
4. Google Summer of Code
5. Open Questions
6. Summary and closing

Notes

Attendance

• tclarke
• kstreith
• gmartin_cn
• dsulgrov
• dadkins
• rforehan
• raevans
• lakhsup (partial)
• abaghda (partial)

Summary

This meeting dealt primarily with Google Summer of Code questions. Clarification was provided on a number of project ideas including image stacking and dependency management. Details can be found in the meeting log.

Decisions

Logs

2010-04-05T10:58:51 <tclarke> ok, let's get started with the meeting
2010-04-05T10:58:58 <tclarke> we'll begin with attendance. If you are attending the meeting, reply with "here"
2010-04-05T10:58:58 <tclarke> here
2010-04-05T10:59:01 <kstreith> here
2010-04-05T10:59:01 <gmartin_cn> here
2010-04-05T10:59:03 <dsulgrov> here
2010-04-05T10:59:09 *** dadkins has joined #opticks
2010-04-05T10:59:11 <dadkins> here
2010-04-05T10:59:18 <rforehan> here
2010-04-05T10:59:25 <tclarke> 1. Attendance
2010-04-05T10:59:25 <tclarke> 2. Announce agenda
2010-04-05T10:59:25 <tclarke> 3. Approve March 22, 2010 Meeting Notes. Ask for review and address any problems seen in the notes.
2010-04-05T10:59:25 <tclarke> 4. Google Summer of Code
2010-04-05T10:59:25 <tclarke> 5. Open Questions
2010-04-05T10:59:25 <tclarke> 6. Summary and closing
2010-04-05T11:00:37 <tclarke> meeting notes from last time are here: http://opticks.org/confluence/display/opticksChat/March+22%2C+2010+Meeting+Notes
2010-04-05T11:00:41 <tclarke> if you notice a problem, please let me know
2010-04-05T11:00:47 <tclarke> no response by the end of this meeting will indicate acceptance of the notes
2010-04-05T11:00:54 <tclarke> the main point for discussion today is Google Summer of Code
2010-04-05T11:01:00 <tclarke> we've had quite a few students interested and have received a couple of proposals so far
2010-04-05T11:01:06 <raevans> here
2010-04-05T11:01:13 <tclarke> main areas of interest seem to be HSI algortithms, SAR algorithms, astronomical processing, and image stacking
2010-04-05T11:01:23 <tclarke> there's been lots of discussion on IRC and the mailing list which I won't repeat for the meeting
2010-04-05T11:01:31 <tclarke> but I'd like to open up the meeting for any questions, concerns, etc. regarding any of the SoC topics
2010-04-05T11:01:32 <tclarke> .
2010-04-05T11:02:57 <tclarke> one of the biggest questions so far seems to be re: image stacking
2010-04-05T11:03:03 <tclarke> there seems to be quite a bit of confusion as to the nature of this project
2010-04-05T11:03:05 <tclarke> I'll briefly summarize
2010-04-05T11:03:14 <tclarke> Opticks currently supports multi-band images
2010-04-05T11:03:18 <tclarke> those bands are generally in 1 file
2010-04-05T11:03:24 <tclarke> if they are a raw format, we can load from multiple files with 1 band per file
2010-04-05T11:03:36 <tclarke> this capability does not extend to multiple files when those files are not raw
2010-04-05T11:04:54 <tclarke> the most common example is with Landsat data...the most common source for Landsat data ships multiple GeoTIFFs with 1 band per file
2010-04-05T11:04:59 <tclarke> this project would implement this capability
2010-04-05T11:05:10 <tclarke> the suggestion so far is to start by combining multiple (already loaded) 1 band images into a single multi-band image
2010-04-05T11:05:24 <tclarke> once this is accomplished, the information learned about Opticks can be used to implement multi-file BSQ imports with arbitrary file types
2010-04-05T11:05:25 <tclarke> .
2010-04-05T11:07:21 <kstreith> reminder to all SoC students that you can submit your proposal today to the SoC site and continue to work on it until the deadline Friday
2010-04-05T11:07:32 <kstreith> you are permitted to edit it on the site after you initially submit it
2010-04-05T11:07:48 <rforehan> tclarke: Might consider also modifying the EnviImporter to recognize ENVI metadata file type - header with list of individual files for each band.
2010-04-05T11:07:53 <kstreith> in fact this allows us to more directly send you feedback about your proposal
2010-04-05T11:09:15 <kstreith> we do realize for some of you this is the first time you've written up a project proposal
2010-04-05T11:09:23 <tclarke> that is a related possibility but not the main desired result of the advertised project
2010-04-05T11:09:47 <tclarke> any other SoC comments?
2010-04-05T11:09:49 <tclarke> or any open comments?
2010-04-05T11:09:56 <lakhsup> Hi. I needed some more info on 'Dependencies management system' project
2010-04-05T11:10:02 <kstreith> ok, I'll summarize
2010-04-05T11:11:11 <kstreith> we have a number of third party C/C++ libraries
2010-04-05T11:11:19 <kstreith> when we went open-source we hand-spun a dependency management system
2010-04-05T11:11:38 <kstreith> https://opticks.ballforge.net/source/browse/opticks/trunk/future/Dependencies/
2010-04-05T11:11:42 <kstreith> it is a Python script, dependencies.py
2010-04-05T11:11:49 <kstreith> with an individual Python script per dependency
2010-04-05T11:12:01 <kstreith> and basically our Dependencies script manages download and unpack of each third-party dependency
2010-04-05T11:13:20 <kstreith> we pre-build them and bundle them into archives (.zip, currently)
2010-04-05T11:13:30 <kstreith> we are looking to replace this for a couple of reasons that tclarke will explain
2010-04-05T11:13:37 <tclarke> we have a number of deficiencies
2010-04-05T11:13:45 <lakhsup> like?
2010-04-05T11:13:48 <tclarke> currently, we pack binaries for all supported platforms in a single zip...this generates large downloads which are not often needed
2010-04-05T11:13:57 <tclarke> we also have external extensions which used the downloader...these are downloaded to the same directory as the core dependencies
2010-04-05T11:14:03 <tclarke> which means you can't use one checkout of the core with multiple checkouts of the same extension
2010-04-05T11:15:28 <tclarke> there's also no way to handle transitive dependencies...they are currently flattened to first level dependencies
2010-04-05T11:15:49 <tclarke> there's no way to handle dependency download mirror sites so everyone is downloading from the same server
2010-04-05T11:15:51 <tclarke> finally,
2010-04-05T11:16:05 <tclarke> you can currently download all dependencies (the default) or a single dependency at a time bu there is no convenient way to specify dependency subsets
2010-04-05T11:16:12 <tclarke> which is useful for plug-in developers who typically only need 4 or 5 of the dependencies, not the whole set
2010-04-05T11:16:12 <tclarke> .
2010-04-05T11:16:18 <lakhsup> so we want to segregate the core dependencies from the rest.. and use a tool to manage them during install
2010-04-05T11:16:23 <tclarke> yes, that's part of it
2010-04-05T11:17:45 <tclarke> we are currently leaning towards using a tool called ivy (part of the apache project)
2010-04-05T11:17:51 <tclarke> but this is a recent development so we are open to other suggestions
2010-04-05T11:18:05 <tclarke> the project would be to configure ivy to work with our dependencies and write decompression plug-ins for ivy (so there's no need to also use apache ant for the decompression)
2010-04-05T11:18:15 <tclarke> or, if there's a better system available, use that instead of ivy
2010-04-05T11:18:16 <tclarke> .
2010-04-05T11:18:23 <lakhsup> I will try to research other alternatives to Ivy and list their pros and cons in my proposal, so we can have a picture of what'll be best for us
2010-04-05T11:18:38 <tclarke> we've looked at a custom system using scons as a basis, we've looked at maven, also looked at OS level systems such as ipkg
2010-04-05T11:19:50 <tclarke> one other requirement is that this work on multiple platforms
2010-04-05T11:20:03 <lakhsup> ok
2010-04-05T11:20:03 <tclarke> Solaris 10 (SPARC), Linux x86_64, Windows 32-bit (XP and Vista), and Windows 64-bit (XP and Vista)
2010-04-05T11:20:18 <tclarke> any further questions or comments?
2010-04-05T11:20:18 <lakhsup> In view of the short time to submit a proposal, I think I'll focus on Ivy
2010-04-05T11:20:20 <abaghda> Hello, I have submitted a project proposal yesterday related to octave algorithm scripting. Could you please give me a feedback ?
2010-04-05T11:20:29 <kstreith> abaghda: i did earlier today
2010-04-05T11:20:34 <kstreith> you may need to login to SoC site to see it
2010-04-05T11:20:43 <kstreith> i noticed they require you click a button to "subscribe" and get auto-notified of changes
2010-04-05T11:22:06 <abaghda> oh, I didn't know that
2010-04-05T11:22:09 <lakhsup> thanks tclarke and kstreith.
2010-04-05T11:22:13 <kstreith> yeah, neither did I
2010-04-05T11:22:20 <kstreith> that's why it took me a day and half after you submitted to notice your proposal
2010-04-05T11:22:24 <kstreith> I expected an e-mail from the SoC site as well
2010-04-05T11:22:53 <tclarke> ok, open questions
2010-04-05T11:22:56 <tclarke> if there are no more, we'll finish up
2010-04-05T11:24:22 <tclarke> ok, thanks for attending. I'll post meeting notes and the next meeting's agenda to the dev mailing list