User:Ppawel/ActivityStream

Problem

Nicely outlined in http://brainoff.com/weblog/2012/03/30/1773 for example.

Solution

Implement activity stream UI for user pages, see http://ui-patterns.com/patterns/ActivityStream/ for detailed description. This will give more engaging experience.

Functional design

High level

Taken from http://brainoff.com/weblog/2012/03/30/1773:

• Recent mapping activity in multiple areas you’re interested in. Should be higher level than individual changesets (SteveCoast has made 12 edits over the last week in Seattle), but allow you to dig in further if you want.
• Some visual distinction between trusted vs non-trusted
• Also deal better with “big” edits, bots, etc. Filter these, or make them visually distinct.
• New mappers in these areas, and their activity. Highlight these, it’s a good opportunity to reach out.
• Highly trusted or experienced mappers in these areas, and their activity. Again, sometimes highlight, sometimes filter, depending on the need.
• Non-mapping activity in the area … upcoming mapping parties, other local news.

Other:

• Allow commenting on activities (or at least meaningful ones like changesets)? That would greatly improve feedback process - no need to write personal messages with links to specific changes/ways - context is right there in the activity.
• "Like this edit"? Maybe not...

Types of activities in the stream

• You modified Paper Street via web editor
• Mark joined your area as a mapper
• Greg modified a bridge in Simpletown
• Betty wrote new diary entry Why I love the new activity stream feature
• ...?

Changemonger

• Very interesting new project: https://github.com/emacsen/changemonger
• Changemonger can generate a description for a given OSM-related object (node, way, relation, changeset - or a collection of those) - which is exactly what is needed for the activity stream (or parts of it at least).
• Specifically, Changemonger could generate descriptions for the following activities:
  • Recent changesets of nearby users
  • Recent changesets of friends
  • Recent changesets of a user (when looking at some user's profile - their recent edit activity)

Technical design notes

Option 1: dynamic

About

• Activity stream would be generated on-the-fly from existing data when user page is rendered.

Pros

• No additional data in the database.

Cons

• Displaying user activity requires various additional queries (get nearby users, their edits etc.)
• Pulling data into the stream from external services may be tricky.

Option 2: static

About

• Activity stream gets its own data model which is persistent, e.g. creating/closing a change set causes Activity instance to be created and saved.
• Displaying user activity is simply listing Activity instances.
• Activity instances don't have to live forever - there could be a limit, e.g. "200 per user" or "only last week" so maybe data redundancy is not THAT bad?

Pros

• More flexibility - can add different types of activities easily by extending the model.
• Can easily feed data into the activity stream from external sources/servers - harder to do with the dynamic design variant.

Cons

• (Potentially) (a lot of) additional data/overhead in the main database.

Option 3: external

About

• Activity stream is generated on and served from a server separate from the main application server.

Pros

• No need to modify the Rails port other than adding some view stuff (HTML/JS for displaying the stream on user page).
• More flexibility for the activity "server" architecture - can be static or dynamic itself since it's completely decoupled from the main server.

Cons

• Adds complexity to the logical architecture - another logical component that is a dependency and a point of failure (activity server is down = no activity stream updates).
• Adds complexity to the physical architecture - another server/database to deploy/maintain.

Integration with Changemonger

• It's in Python... so using it as a library would mean writing the whole thing in Python (which basically requires the "external" design option - because the main Rails port is in Ruby so Changemonger couldn't be easily used as a library).
• Using Changemonger as an external service would require a running, up-to-date and highly available Changemonger instance.

Technical design (final version) (work in progress)

High level architecture

Open questions

• Isn't Changemonger <-> OSM public API communication too expensive?
  • In pessimistic scenario, for each changeset it will make two calls plus X calls where X is equal to the number of objects in the changeset.
  • One option is pack all changeset data (objects, old objects) into the event emitted from the Rails Port to Activity Server - then Changemonger can do its thing without calling the API (emacsen to verify this statement).

Events

Type	Origin	Data required to generate an activity stream item	Users who will get the item
New user in the neighborhood	user_controller.rb	New user data	Users nearby to the new user Friends of the user
New diary entry	diary_entry_controller.rb	User data Diary entry data	Users nearby to the author Users nearby to the diary entry location Friends of the author
New changeset	changeset_controller.rb also need to figure out how to handle implicitly closed changesets (timeout)	Changeset data Changeset objects data (ways, nodes, relations) Data of untagged objects (needed by Changemonger)	Users nearby to/within the changeset bbox Users nearby to the changeset author Friends of the changeset author

Activity server internal design

Open questions

• What technology is used for storage?
  • Postgres?
  • http://incubator.apache.org/kafka/ looks interesting for this purpose.
• How does the Activity Item Generator get the data and how does it know what users are interested in what activity items?
  • Should the events incoming from the Rails Port include the data?

Notes / random stuff

• User page redesign work already done: https://github.com/openstreetmap/openstreetmap-website/pull/23
• Social features wish list: http://brainoff.com/weblog/2012/03/30/1773
• LinkedIn activity stream uses http://incubator.apache.org/kafka/
• Some useful SO questions/answers:
  • http://stackoverflow.com/questions/202198/whats-the-best-manner-of-implementing-a-social-activity-stream
  • http://stackoverflow.com/questions/1443960/how-to-implement-the-activity-stream-in-a-social-network
  • http://stackoverflow.com/questions/762490/how-do-social-networking-websites-compute-friends-updates

Status

• I am looking into existing work on Github, merging it into current master to see it in action and hack on it
• Rebased current work on user pages to current master - https://github.com/ppawel/openstreetmap-website/tree/activity-stream
• Implementing (for now with design Option 1)
• Created pull request on Github - https://github.com/openstreetmap/openstreetmap-website/pull/54
• 2012-08-22: Still in research/design phase trying to figure out how to approach the implementation.