Skip to content

Why and How Networks Connect

I started the netfoundry project because I had been in correspondence with several people over the years about the need for a “network-of-networks” type of social media/organizing infrastructure. To further elaborate on what netfoundry is intended to accomplish, we need to first examine why networks connect. Then we can discuss how networks connect.

Why Networks Connect

People form networks in order to fulfill some need(s). There may also be a stated mission or goal, but these are also driven by needs. One network connects to another in order to get help in fulfilling needs. Usually the relationship is mutually beneficial such that the network providing the help benefits in some way through providing it, perhaps through some exchange of resources, knowledge, physical presence, publicity, shared mission fulfillment, etc. The partnership may naturally dissolve once the needs are met and the exchanges are complete.

Today’s Social Networking Platforms are Inadequate

Internet based software systems are an obvious means of connecting networks together. However, there is a glaring problem with the approach social networking platforms have taken for allowing constituents to connect and express their needs through “status updates”. The problem is that status updates are typically free form blobs of text. There are usually a few metadata fields associated such as location and date, but the substance of the update is opaque to other systems. This is fine for casual social interaction, but if you’re trying to get something done, or communicate something important such as to request the fulfillment of a need or to offer a skill or a resource to your network, it’s like shouting into the wind. In a matter of seconds after posting, your update will drown in a deluge of updates from the rest of the world.

By simply pulling out some basic metadata from the status update, we can create a system in which important updates endure for a longer time period and become machine readable by other systems and networks. This is essential to promote cohesion, mutuality and emergent behavior in a network.

Participant Defined

Let’s say we have a small group of individuals collaborating on some project or undertaking. Let’s call it a “NetNode” and identify its constituent parts as People, Needs, Assets, Connections and Events. This group might publicize its attributes to the wider world to create the potential for connection and collaboration with other groups. Let’s represent this atomic participant visually as Figure 1. Note the purple box on the right side. This is meant to represent a point of connection with other NetNodes, as we will see shortly.

netnode

Figure 1: NetNode

Collaboration and Encapsulation

Now we can begin to connect these groups together. The nature of the collaboration should allow for win-win mutuality (needs matched with assets/skills) to flow effortlessly between the participating NetNodes. The collective entity that forms out of such collaboration is not, in principle, different from the participant represented in Figure 1. So let’s draw an ellipse around it and represent it as a NetNode, a participant in its own right, as in Figure 2. In other words this new entity also has collective People, Assets, Needs, Connections and Events and can publicize collective attributes for connection and collaboration with further networks. Again, note the purple box on the outside of the ellipse.

netnodecollab

Figure 2: NetNode Collaboration

Recursive Nesting

Now the organization pictured in Figure 2 might collectively decide to bring in another partner network. They two groups that make up Figure 2 might decide to individually collaborate with the new partner. But for the purposes of our example let’s say they decide to interface with the new partner collectively. Then we’d get Figure 3. Again the collaboration represents another NetNode with the same principles as the any other, including the potential to connect as a whole to other NetNodes, i.e. the purple box outside circle.

In a real world example, imagine a consortium of writer’s guilds collectively deciding to network with a publishers association. Or perhaps a group of festival organizers wants to collaborate with a musicians collective.

netnodecollabnesting

Figure 3: Nested Collaborative Networks

OK now take this idea to the Nth level and you get a fractal-like nested organization of networks, such as in Figure 4. Each group is in principle the same as its constituent parts in that there is a stated purpose, win-win mutuality and communication. And there is also the potential to connect as a whole to further networks, again as indicated by the purple boxes on the sides of the figure.

netnodehappynetwork

Figure 4: Happy Fractal Networks

Now that’s what a happy network looks like!

Principled Approach

Now apply the Core Principles to that picture. Participation in such a structure entails no loss of autonomy and no cooptation. A node is as powerful as the information flowing through it to/from other nodes, which is a function of how much it is trusted and how well it facilitates matching needs to capabilities across the network.

Next Steps

This post has described one way in which networks can connect. In software systems a “way of connecting” is accomplished through an Application Programming Interface or API. Participants have to agree to use the API and to communicate using standard data formats, analogous to speaking the same language. The next step is to begin to establish the API and data formats, as well as the standards management organization that would see to their equitable maintenance and evolution. Interested in taking those next steps? Get in touch!

Core Principles

What is a Collaborative Network?

These days we throw around the word “network” like it has a given meaning that everyone can agree with immediately. But if you dig into the word, and survey the terrain of the network world, it becomes clear that not all networks are created equal.

Many networks today are really just proprietary applications. For example, most social networks are built within the confines of a corporate structure, membership is controlled by that structure and all the behaviors of the network are carefully scripted and governed. Ask Facebook if they would be okay with you spinning up a new instance of their application on your web host. Good luck with that!

The mother of all computer networks, the Internet, is beautiful work of collaborative engineering. Each layer of the network is designed to encourage maximum coherence and inclusion among participants. Its standards and technologies undergo a large measure of peer review by bodies such as the W3C for example, before they are adopted and promoted. The result is that everyone agrees on how the network works, and one does not have to engage the IETF in order to set up a new website, or a VPN, for example. The Internet is a collaborative network. It even resists efforts of participants to co-opt the network for centralized control.

A collaborative network is one that removes barriers to entry, facilitates communication based on standards, discourages centralized authority and promotes organic growth. If you can add a new node to the network without re-engineering and without asking permission from some central authority, and if that new node can immediately begin transacting with other nodes, and if the network exhibits behaviors that could not easily have been specifically preconceived and designed, then you have a collaborative network.

Core Principles of Collaborative Networks

Let’s look at what might be the core principles of such networks. These principles should speak to the values, or culture, of the network rather than create a set of prescriptions that must be followed. If participants on the network share these values, then the network will flourish.

Open APIs

This should be obvious. Getting on the network requires computers to interoperate. If the APIs of each network node are not open to participants, then there is no network.

Standard Data Formats

Everyone needs to talk the same language in data exchanges, and participants must actively engage in language maintenance and evolution.

No Command and Control

Network nodes function autonomously based on information available. They may share information with other nodes but cannot expect or require specific action to be carried out in response to said information sharing. “Keep me informed, but don’t tell me what to do!”

Flexible Data Model

Expect change. Build all systems with sufficient abstraction from data models. Collaboration means building off of what others have done. It means efficiently finding solutions together. Locking down aspects of the system prevents flow and signals the onset of rigor mortis.

Self-Organization

There is no central authority in a collaborative network. Participants decide with whom to connect and with whom to share information. Each participant is autonomous and equally empowered. The network provides tools to facilitate node discovery, connection and growth.

Emergent Behavior

The behavior of the network as a whole should not (and cannot) be a preconceived design, rather it should be allowed to emerge from the autonomous functioning of all its parts. Releasing the constraints of an upfront design promotes novel and unexpected behavior.

Trust

Collaboration requires it. The participants in a collaborative network are there to work together toward a common vision. Those who reveal themselves as not being benevolent toward the vision will likely find themselves cut out of information sharing relationships, and in a collaborative network lack of information flow to a node results in the death of that node. Thus, the more trusted a participant the more information available to it and the more value it can bring to the network as a whole.

Building the Network

netfoundry is designed as a progenitor of networks that embody the above principles. In such networks the participants are what they need and what they share, and the more they share the less they need. If you are interested in building this kind of a networked world, get in touch!

What Is netfoundry

Imagine If You Will

You’ve got a great new software idea in which you’ll build cool pieces of functionality that will allow end users to do amazing things and developers to build exciting applications against your APIs. Your application will be the next big thing. You build it. Users show up and request all kinds of great new features. They want the pieces of your application to interact in ways you did not consider. Developers show up and request access to the guts of your system so they can fine tune you and control you.

Let’s turn this model upside down. Let the objects of your system interact with each other based on their connections to each other rather than on some predetermined orchestration. Let the outside world be informed when something in your system changes, and vice versa, but do not allow them to directly invoke your functionality.

What you lose is tedious hand stitching of complex behaviors and intrusive command, control and integration of outside parties. What you gain is self-service, self-organization, an organic software ecosystem that spills over into external software applications and produces surprising, emergent behaviors.

Enter netfoundry

netfoundry is an open source node.js based platform for self-organizing collaborative object networks. Objects link to each other, share state and notify each other of state changes. All interactions are event driven and queued for scalability. Object state is persisted in a NoSQL database. netfoundry is based upon ideas from the Object Network and FOREST projects of Duncan Cragg.

With netfoundry you get a solid framework for managing object state and messaging between linked objects in your system. And since links are just represented as URIs, you can link to external objects and notify them of your state changes via RESTful services.

All of this rests upon the agreement between participants of the format or schema of the object state. Participants do not have to agree on what functionality is triggered when a state change message is received. That’s up to the receiving party. And the receiver will probably change state as a result, which will trigger state change messages to its subscribers. And so on. Interesting behavior emerges from the network, not from the individual nodes.

An Example

I schedule an event in my cool new calendaring software. Within the event details I specify that I need a video projector at the event, but I don’t have one handy and could someone please bring one.

I invite a bunch of organizations to the event, and each one that accepts the invite receives a copy of all the event details. One of those organizations sees that I need a video projector and has one available to bring to the event. They create a link between the video projector asset and the event they received. This link request is sent to me. I confirm that link and celebrate that I don’t have to go buy an expensive projector.

Note that in such a system, no one has to orchestrate a functional interaction of borrowing a video projector. That behavior emerges from the expressed need of the event organizer and the expressed generosity of the event invitee. The participants in such a network only have to make their state known to those with whom they are linked, and to request new links as appropriate to enrich network interconnectivity.

Now What

It’s early days for netfoundry. I’ve just posted the framework and a few working examples to the netfoundry github repository. Developers, feel free to take a look and fork it. I’d love to have your feedback and participation.