Everest v.0

The original Everest - the iteration underpinning PlanetEngine - was deliberately monolithic: a single software package handling a wide range of tasks common to all kinds of modelling workflows.

During the pandemic however, three things became apparent. The first was the broader utility of the individual components of Everest in and of themselves: for example, its flexible and versatile visualisation code, or its extended Python classing tools. The second was the larger-than-expected volume of boilerplate code necessary to make new engines useful to end users, pointing to missing common functionalities that could be gainfully incorporated into Everest proper. The third was the surprising relevance of aspects of the Everest project to various emerging themes in the broader tech sector, from blockchain to decentralised finance and artificial intelligence.

It became clear that Everest could benefit from a more modular design, exposing the capabilities of its various facets and establishing a template for the introduction of new facilities beyond the original scope of the project. This in turn pointed to a need for both higher abstractions and deeper common structures. In short, there was a clear argument for making the Everest ‘pyramid’ simultaneously wider, taller, and richer.

The new design of Everest, pictured below, takes the form of a suite of interoperable but independent tools sharing a common framework and cumulatively supporting a complete modelling workflow from construction to publication.

Some components have undergone name changes.

These are the components of Everest v.0, now in active development.

Ptolemaic¶

‘Ptolemaic’ is Everest’s core object model. It provides a root class of immutable, hierarchical data objects called ‘Ptolemaics’ or ‘Ptols’, which can be either Primitives - encoding irreducible notions like quantity and duration - or Composites - encoding meaningful collections of Ptolemaics brought together under a given context. The Ptolemaic system as a whole therefore understands all data as belonging to a single universal tree of representations. Each Ptolemaic is endowed with an interface derived from the position of each Ptolemaic on the tree, from which input types and output types are inferred, giving the whole system the character of an algebraic type system. Ptolemaics are meant to be viewed as universal: a given Ptolemaic constructed in one place and time should be guaranteed to function the same as one constructed in another place and time. They are also designed to be reducible to very concise string-based encodings called Epitaphs. These features of Ptolemaics form the basis of Everest’s mechanism for guaranteeing reproducibility, analysability, and portability for all kinds of models.

Gnostic¶

‘Gnostic’ is Everest’s abstract computational model. It is a programming language and a kind of automaton, similar to a Turing Machine, but dealing directly in the Everest paradigm. Its purpose is to provide a common reference point for different models, allowing us to reason more deeply about what a given piece of code in Everest is meant to do. Gnostic functions a bit like an algebra over filepaths, with a single computational ‘action’ - simply ‘go’ - that is conceptually akin to following a shortcut in a file tree. By providing a single, unchanging ‘canonical form’ of every program, Gnostic essentially future proofs Everest for all time, as - in even the worst case scenario - any two programs written in any two languages by any two people in any two places and times will still be able to ‘speak’ to Everest by translating themselves into Gnostic first.

Uniplex¶

‘Uniplex’ is Everest’s database system. Uniplex is a close twin of Ptolemaic, but instead of implementing a representational tree, it implements a filetree. Uniplex defines the ‘Plex’ file type: a high-performance monolithic database file based on H5. The Plex is made up of ‘Plexons’ in the same way that Ptolemaic is made of up Ptols. Plexons come in a variety of ‘Plextypes’, which function analogously to the data types and array shapes of conventional tabular data systems but with broader capabilities. Every valid address inside a Plex database corresponds to a unique Ptolemaic object, essentially allowing Ptols to be directly read from and written to an attached Plex file. Each Plex is headed by a Manifest, to support accelerated access, as well as a Preamble, which provides concise top-level encodings for arbitrary objects according to user discretion. The Preamble is particularly important because it allows each particular Plex file to be as light as possible while still remaining self sufficient, enabling Uniplex’s biggest trick: the Uniplex itself, which is conceived as the set of all Plex files in existence; in essence, the Uniplex is a single shared file system for the whole human race.

Symphony¶

‘Symphony’ is Everest’s orchestration framework. Its purpose is to form the bridge between code and compute, separating these in a unique manner that is ideal for Everest’s decentralised idiom. The core idea of Symphony is the ‘Genie’. A Genie is the opposite of a ‘daemon’: whereas a ‘daemon’ is a persistent process that controls other processes, a Genie is an ephemeral process that is triggered and controlled by others. Worker nodes belonging to the central node’s given ‘explicit trust network’ are permitted to tunnel into the central node and run the Genie locally, but under their own authority. Typically, the functionality offered by the Genie will involve the distribution and collection of tasks: by delegating these tasks to a perpetually vast and branching network of workers, a light central node can lift a heavy computational load. In this sense Symphony functions a bit like BitTorrent, except with more work - and more authority - delegated to the peers.

Quest¶

‘Quest’ is Everest’s networking tool. It is an alternate application layer protocol: essentially a World Wide Web for modelling. Quest provides the connective tissue that allows other Everest features like Uniplex and Symphony to range across the Internet. It provides protocols for extending ‘explicit trust networks’ beyond institutional boundaries and for managing diffuse data flows whether large or small. Quest is intended to challenge blockchain as an alternative foundation for ‘Web3’, drawing inspiration instead from the self-hosted web of the 90s and the filesharing movement of the 00s. Eventually, the ambition is for Quest to provide the backbone of a single, addressable, programmable planetary computer.

Vertex¶

‘Vertex’ is Everest’s visualisation suite. Descended from the original Everest’s ‘Window’ module - an object-oriented wrapper around Python’s ‘matplotlib’ - Vertex goes much further by defining a single overarching visual concept: the ‘Aperture’. An Aperture is a data object representing a view of a given object from a given perspective and within a given context. Apertures can be nested and combined in a variety of ways. Archetypally two-dimensional, as the name implies, the Aperture paradigm actually goes beyond this, extending to arbitrary dimensions including time, sound, text, and interaction. Apertures are intended to offer ‘views’ of Ptolemaics, but also are themselves Ptolemaics, meaning they can be stored in a Plex or made the subject of other Apertures (in a sense allowing ‘visualisations of visualisations’). Aperture eschews any dependency on pre-existing visualisation codes in favour of browser-native Web standards like HTML, CSS, SVG, and WebGL. Because its interface languages are web-based, Aperture objects served through the Quest protocol essentially function as Quest’s equivalent of web pages, completing the triad of capabilities that make Quest a fully-featured alternative to the World Wide Web.

Bureau¶

‘Bureau’ is Everest’s work environment software. Bureau connects the other Everest services to the local machine in the same way that Quest connects them to the Internet. It is the bridge between the temporally and spatially bound world of a human user with the timeless and placeless world of the Ptolemaic data model. Bureau wraps around Ptolemaic objects to service the many features that make them practically useful: visualisations using Vertex, data storage using Uniplex, compute via Symphony, and more. Naturally, Bureau objects are themselves Ptolemaics, making the user’s work environment just another object that can be saved, loaded, backed up, distributed, analysed, or automated.

Ingenue¶

‘Ingenue’ is Everest’s in-house software development suite. Its primary function is to make it easier to build the Everest ‘Engines’ that are the primary way Everest makes itself useful and valuable to so many different disciplines. Since all Everest programs are Everest (Ptolemaic) objects in themselves, Ingenue is in a certain sense a way for Everest to reason about and develop its own capabilities. Ingenue supports Git-like version control as well as Copilot-like metamodelling, and integrates with Uniplex and Quest to provide a channel for publishing new Engines fully within the Everest ecosystem. Its essential purpose is to nurture an Everest way of writing and thinking about software.

Horizon¶

‘Horizon’ is where it all comes together: binding all the different functions of Everest into a single piece of software that is quite simply an ‘operating system for science’. Written in WebAssembly - the ‘assembly language of the Internet’ - and thus universally compatible across all hardwares, from desktops to smartphones, Horizon will be simple, beautiful, relaxing, immersive, and 100% distraction free: a compelling and inspiring environment for the difficult work of discovery.

Engines¶

The tools listed above will collectively make up the ‘Everest Core’: the foundational technologies that we think are universally relevant to all modelling, no matter the discipline. Wrapped around the ‘Core’ are what we call ‘Everest Engines’, which are extensions to the Everest codebase, built using Ingenue, that make it specifically useful to a particular discipline. PlanetEngine, for example, will the part of Everest that tackles planetary modelling problems. Finally, wrapped around the Engines will be Everest Applications: nice, friendly, web-based interfaces for Engines built using Vertex and hyper-specialised for particular use cases. Rounding out the collection will be Everest Kits: useful extra bits and pieces that are neither specialised enough to be incorporated into Engines nor general enough to be included in the Core. This will help us keep the Core as lean and robust as possible while simultaneously providing a channel for the most promising new code to contend for inclusion.

Several Everest Engines have already been built and are supporting real-world research problems. Learn more.

What’s next?¶

The Everest project is without a doubt hugely ambitious, but mercifully, it doesn’t all have to be accomplished at once. By focussing on essential functionality first, and compartmentalising the codebase for incremental development, we can ensure that Everest is useful today, not in the distant future. If we build it right, we can lay a foundation that will deliver the crucial fifty years of revolutionary progress we need to survive the age of crisis - and build a better world.