I'm trying to do some analysis (ideally visual analysis!) of networks of order 1 million nodes and 10 million edges.

I normally use Gephi, but for this size network it seems to struggle (unless maybe I need to up the size of my machine and allocate ever large amounts of memory to Java? Is there a rule of thumb for calculating how much memory to allocate? Do net analysis tools like Gephi try to keep the network in memory?)

A couple of folk have recommended Cytoscape (I'm downloading it now!) but it also seems to to be Java based?

Any idea what size networks NetworkX is happy with?

Are there custom libraries around that work well with large networks? What languages are best suited to working with large networks?

asked 20 Jun '11, 18:31

psychemedia's gravatar image

psychemedia ♦♦
1.1k323961
accept rate: 11%

Java applications are often given a heap size on start up as a command line flag. This could be what's hurting Gephi.

(10 Aug '11, 03:45) timClicks ♦♦

I've found that NetworkX seems to be able to deal with large graphs very well. It is limited to the size memory available on the machine. However, layout algorithms are CPU bound.

link

answered 10 Aug '11, 03:48

timClicks's gravatar image

timClicks ♦♦
16346
accept rate: 0%

This doesn't relate to the performance, per se. One other point: I've found adding lots of properties to nodes to be fairly unwieldy with NetworkX. I've taken to only storing a reference in each node. When I want the properties of an object, I call a row in a table.

(10 Aug '11, 03:48) timClicks ♦♦
Your answer
toggle preview

Follow this question

By Email:

Once you sign in you will be able to subscribe for any updates here

By RSS:

Answers

Answers and Comments

Markdown Basics

  • *italic* or _italic_
  • **bold** or __bold__
  • link:[text](http://url.com/ "title")
  • image?![alt text](/path/img.jpg "title")
  • numbered list: 1. Foo 2. Bar
  • to add a line break simply add two spaces to where you would like the new line to be.
  • basic HTML tags are also supported

Tags:

×3
×2

Asked: 20 Jun '11, 18:31

Seen: 842 times

Last updated: 05 Jul, 04:08

powered by OSQA