Re: [Visualpython-users] VPython for Graph Visualization

On Tue, Jul 10, 2012 at 4:29 PM, Mark Adam <dre...@gm...> wrote:
>
> I'm wondering if there's been anyone who's explored using the 3-d
> visualization extension of python for graph visualization?

ha, well, i'm actually giving a short presentation/demo of exactly
this at work tomorrow for ~40 or so other developers, and, in
retrospect ... all of you ;-)

i can't really share the code ATM (it belongs to me but currently
integrates with the production AMQP network, etc) but i do intend on
releasing at a later time.

tbh, it really is just a demo at this point, but still rather
impressive (many oohs and ahhs) considering a SLOC barely over 200 --
strongly hinting at vpython's power-thru-simplicity model.

the talk itself focuses on the natural intuitions behind generic 3D
interfaces, and self-organizing, real[ish]-time visualizations (sadly,
for both, sans some sweet HID equip + libs needed to move from dream
-> reality -- eg. Xbox Kinect sensors4all -- kbds and mice are
OBSOLETE! yuck! :-)

SOURCE: application binds to message queue, subscribing to real-time
phone-state queues (emitted by ~400 CSR agents in another facility,
across the country [US]). these are nothing more than (time, agent,
state); other properties are available but unused.

INTERFACE: outside "observers" (ie. users) may navigate this universe:
 - free-space navigation
 - click-to-mount camera (to an arbitrary sphere)
... or influence fundamental constants/properties as deemed useful:
 - gravity
 - intial vector
 - intial vector magnitude
 - various constants like growth/etc
... modifying the output + user perception/experience stream, and
[potentially] surface new insights. introspection/aggregation
[stats/avg/etc] on individual agents and/or logical groupings
contribute usefulness as a human interface (think current-get FPS
games, and the massive information I/O densities in effect).

SELF-ORGANIZING: to complement humans already superb pattern matching
abilities, the visualizer renders agents as simple, colored spheres
orbiting a gravity-well anchored to origin. state change == color
change. agents grow at a continuous pace while idling in one state;
any transition resets the radius back to 1. from just these basic
"truths", patterns emerge like:
 - "seems like ~40% in idle at any given time"
 - "why are so many busy/unavailable [red]?"
 - "why agent XYZ in red [busy], and 3x larger?"
 - "technical problem? everyone just went IDLE"
... what's truly incredible is the fact that near *everyone*
interprets the output similarly, by virtue of a shared experience,
that of life (another variant employs magnetic-like interactions
between agents)

[1/2] tl;dr, physics fits really well in this domain! it's something
we've practice, without choice, since day 1, and are pretty well
disciplined by now :-)

quick aside: at a previous company, we ran gltail [
http://www.fudgie.org/ ] all day, every day, and i can't even begin to
tell you how many times network or completely unrelated *problems*
were spotted by my peers throughout the office -- read: very
NON-TECHIE folk! -- on numerous occasion. often, IMO, the anomalies
were often subtle -- far from approaching boundary alarms a la
Nagios/Cactus/whatever, but still with [complex] negative implications
if left unbound -- these people simply noticed "the blobber feels
off/out-of-sync today".  it was phenomenal ... inspiring continued
interest, and soon after, leading me to [v]python :-)

moving along ... the impl is not too crazy-fancy, but worthy of
mention.  i run the vpython bits under uWSGI
[http://projects.unbit.it/uwsgi/ : absolutely fabulous, check it out]
within an offloaded/independent "--mule" process, avoiding detrimental
conflict with HTTP/WSGI request/response cycles. this is critical, and
allows painless interaction with the vpython/visualizer loop, from the
outside, via any one of the umpteen uWSGI methods. a single FX class
manages transitions thru staticmethod(generator) members.  each agent
sphere carries an "FXQ" [effects queue], filled with LIVE,
instantiated generators.  during each cycle/iteration/Hz, i simply
call next(generator) on each generator, in each queue, thereby
advancing the entire system one unit-over-Hz. FX behave asynchronously
(thx to generator-awesome-ness) and perform everything from applying
gravity -> velocity -> new_pos all the way to managing the camera
updates, timeouts, etc. etc. when a generator expires (raises
StopIteration), it's removed from it's respective FXQ.

the FXQ model its immensely simple, but i suspect not all that
efficient or scalable; better would be some
generic-as-possible-low-level-transform-engine parallelizing some
relationships, eg. via matrices or other transforms ... sadly, despite
insatiable fervor for mathematics and physics, i get paid to architect
web/system/business applications, my computational physics skills
surely suck ... pointer very much welcomed!

... hopefully you found what became a short story either interestingly
entertaining or entertainingly interesting, because The End, is ...
NOW!

* tune in next week for the shocking conclusion! *

-- 

C Anthony