The Vision Technology
Business Entity Applications
Once upon a time, most people assumed that business entities were simple things easily handled using existing techniques and technologies. That was before most people gave much thought to the matter. Our experience working with financial, economic, and market research data, entities and applications, particularly ad hoc, analytic, decision support applications, has taught us otherwise.
Business entity applications involve the integration of substantial amounts of internal and external data. That data is almost universally time varying and governed by non-trivial rules regarding its interrelationships and use. While temporal dependencies are one source of those rules, they are not the only source.
Applications cannot be partitioned neatly into 'the bulk part the database does' and everything else. Applications are collection and query intensive with collection and query operations interspersed throughout an application and across levels of encapsulation and abstraction.
Transactions are long running and ad hoc, with ad hoc explorations often leading to new rules and building blocks. Explorations are conducted and new applications developed by business knowledgeable users, not programmers and 'database gurus'. Those rules and building blocks must be reusable at all appropriate places in new applications. It is unreasonable and undesirable to encumber the productivity of those users or the reuse of their explorations simply because of technological limitations.
When we first wrote this in the early 1990's, the world was beginning to recognize the importance of addressing these issues. Today, these issues are front and center as people work to model and analyze large and growing collections of complex data.
Not An Incremental Technology
The analytic decision support, data acquisition, and data distribution applications supported by Vision in the financial and other information intensive industries are mission critical in their organizations. Vision, a temporal, object-oriented, database management system, has been doing that job successfully, reliably, and with no fundamental change to its architecture or information model since its initial release in 1986.
When we built Vision, we had three significant advantages -- a technological concept based on our experience building data intensive applications and analysis tools, real clients and their applications, and a clean slate. The benefits of a clean slate are obvious. Our clients and their applications provide a built-in validation suite for the technology. That validation is multifaceted: immediate success of the technology's original applications; low development and deployment costs for new applications; low maintenance costs for existing applications; scalable exploitation of the hardware improvements we've all seen over the past decade.
There is no brief way to describe the scope and details of the Vision technology. Vision borrows in many key ways from its database and object-oriented heritage; however, Vision is not an incremental technology. Vision is a database programming language hybrid that implements object-oriented constructs and operations, defined naturally in terms of classes and individuals, in a completely collection centric way. Objects, updates, and computation structures are all represented using a common unifying infrastructure of stateless, abstract objects connected by state and structure bearing morphisms. Both collections of objects and collections of collections are naturally represented and transformed within Vision. Because all operations are implemented as algebraic transformations of underlying object structures and execution state, they are implicitly parallel. This includes all collection operations -- not just selection operations.
Vision's temporal data model is based on early temporal database research such Clifford and Warren's 1983 Formal Semantics for Time In Databases (Vision has been doing this stuff since 1986). In Vision, temporal dependencies are modeled at the level of object properties. Temporal properties can be used explicitly as the time series that they are; however, they exhibit their modeling power when used to select, on the basis of the temporal context of an expression, the property values applicable in that temporal context. Temporal context can be freely bound, rebound, nested, and multiply bound within an expression. The algebra of the Vision execution engine easily parallelizes concurrent temporal contexts so that it is common for one expression to be evaluated for multiple individuals and temporal contexts simultaneously and without restriction. Underlying this model are time series structures with no intrinsic limits on their granularity or regularity and an extensive, extensible collection of date and time classes and operations.
Vision has questioned a number of assumptions and implemented some traditional ideas in non-traditional ways. Physically, Vision objects are not records, and Vision computation does not proceed an individual object at a time. In the examples that follow, if you think you see concrete structures like records and element at a time execution orders, it's all in your mind.
A Modest Example
Account defineMethod: [ | correlatedAccounts | !lowerPct <- pctEq * 0.8; !upperPct <- ...; holdings select: [security type isEquity]. collectListElementsFrom: [security holdings]. groupedBy: [account]. select: [pctEq between: ^my lowerPct and: ^my upperPct]. extendBy: [ !correlation <- groupList correlate: [...] ... ]. sortDown: [correlation] ];
This modest, but typical example, is taken from a mutual fund/portfolio analysis system powered by Vision. It is a skeleton of a method used to order the funds/portfolios in the database based on the degree their holdings correlate with a specific mutual fund (the recipient of correlatedAccounts message). It is typical of the sort of thing that an analyst might need to do at 9 AM for an 11 AM presentation.
While the specific details of the method do not matter, there are a number of key features worth noting:
- Query and collection operations are embedded throughout the logic of this method. Some of those operations are traditional database operations like selection and grouping, others are non-traditional and user defined, such as collectListElementsFrom: and correlate:.
- New collections of information, complete with new properties, need to be created, used, and returned dynamically. This method delivers a collection of results to its caller. It is reasonable to expect that caller to perform more collection operations on the result; it is unreasonable to materialize that collection into an 'outside the database' world that implements those operations in an ad hoc way.
- Collection level detail is just as important as summary information. This is true for both stored database state and dynamically derived information. Here, for example, the Account class has a collection valued holdings property and non-collection valued pctEq property; the groupedBy: operation returns a collection of new objects, each with a single valued reference to an account object and a multi-valued groupList property referencing the elements of the group.
- Much complexity is encapsulated beneath this method. For example, at the level of abstraction of this method, the time variability of both the holdings and pctEq properties has already been properly handled.
MyFunds do: [correlatedAccounts first: 5 . do: [ ... ] ] ^today to: ^today - 1 yearEnds by: 1 quarterEnds. evaluate: [ MyFunds do: [correlatedAccounts first: 5. do: [...]] ] Account defineMethod: [ | displayCorrelatedAccounts | ... correlatedAccounts select: [correlation >= ...] ...; ... ]; MyFunds do: [displayCorrelatedAccounts]