Vision Portfolio Management Application Issues
Overview
Special structures have been designed to manage dividend, price, and adjustment data in the Vision database. These structures have been created to correctly handle split and currency adjustments automatically. In addition, these structures have been designed to facilitate future reorganizations as the database grows, so that applications that use this data are sheltered from structural changes.
Many items defined for a security are affected by stock splits including price, shares outstanding, and dividends per share. By default, data is returned split adjusted relative to the current date. In general, message names that begin with the character '_' indicate the "raw", unadjusted data value in its initial units. For example, the property _sharesOut defined at Security refers to the actual shares outstanding value as of a specific date. The message sharesOut returns this value adjusted for any splits that have occurred since that date.
Split Adjustments
All per share and shares outstanding data in Vision is implicitly adjusted through some date. To analyze per share data as of any date other than this implicit date, it is necessary to account for any splits that have occurred between this date and the analysis date. Two viable approaches are available. One alternative is to actually change the values of existing data to reflect any splits that have occurred. Alternatively, stored data can always remain in its raw, unadjusted form and an adjustment factor reflecting any splits that have occurred can be applied to this data. The advantage of this approach is that data values are never changed directly, greatly simplifying the magnitude of work that needs to be done when a split does occur. In addition, if a split is incorrectly introduced, it is much easier and more reliable to just modify a single adjustment factor than it is to unadjust and re-adjust all of the data that was incorrectly adjusted. Because of the data integrity problems that can arise with the former approach, the Vision database uses the adjustment factor approach in its handling of splits.
To adjust per share data it is necessary to know what splits have occurred between the date the data is known to be adjusted through (adjustment date) and the current date. To accomplish this, Vision tracks splits for each security in a time series called rawSplitFactor. The actual factors are stored in this time series as of their ex-date. For example, if a 2-for-1 split in XYZ Corporation occurred on January 15, 1997, the value 2 would be stored in XYZ's rawSplitFactor as of this date. If XYZ's earnings per share on December 31, 1996 is 10 then the adjusted value of eps on or after January 15 is 5 (i.e., 10 / 2 ). If a 3-for-1 split occurred on March 15, then the value 3 would be stored in XYZ's rawSplitFactor as of this date and XYZ's adjusted value of eps on or after March 15 would be 1.67 ( i.e., 10 / 2 / 3 ).
Vision tracks a second time series called adjustmentFactor that represents the running product of these raw factors. This variable has an initial value of 1. Each time a split occurs, a new point representing the product of the new raw factor and the last adjustment factor is stored in the adjustment factor as of the ex-date. The data from the previous example is illustrated below:
0 1/15/97 3/15/97
rawSplitFactor 2 3
adjustmentFactor 1 2 6
To properly use the adjustment factor data, you actually need to access values as of the two dates involved, the adjustment date and the current date. The ratio of these factors gives you the correct adjustment. Therefore the correct way to adjust per share data that has been adjusted through December 31 for analysis on May 31 is:
adjustment factor ratio = adjustment factor on May 31 = 6
-------------------------------- ---
adjustment factor on December 31 1
per share
adjusted value = unadjusted value / adjustment factor ratio
XYZ's adjusted eps = 10 / 6 = 1.67
The correct way to adjust share outstanding values is to multiply the unadjusted value by the adjustment factor ratio.
The message adjustmentRelativeTo: is used to access the current adjustment factor relative to the date that your data item is known to be adjusted through. If you know that XYZ's eps is adjusted through January 31, then the expression:
Named Security XYZ adjustmentRelativeTo: 9701accesses the value needed to adjust the eps value relative to the current analysis date. To adjust the eps, divide the unadjusted value by this adjustment value:
Named Security XYZ send: [ _eps / # unadjusted
(^self adjustmentRelativeTo: 9701)
]
The previous example assumes that you know the exact date the data
has been adjusted through. You may not always know this explicitly;
however, many data items are adjusted as of the date they become
effective (i.e., the "as of" date).
In this case, the following methodology can be used to access
the adjusted value:
Named Security XYZ getAdjustedDataFor: Named Security XYZ :_eps
In general, methods have been created that automatically return adjusted values by default. For example, the pricing and dividend data described later in this section are adjusted by default. The message _price is defined to return a PriceRecord's unadjusted price (i.e., the original value) and the message price is defined to return a PriceRecord's adjusted price. The message sharesOut is defined at Security to return the split-adjusted value of the _sharesOut property.
Different data sources have different rules for determining the adjustment date of the data. To address the varying requirements, the message adjustmentFactor is normally defined as needed for each class that manages data that can be adjusted.
The message addSplitFactorOf: onExDate: is used to add a new split factor for a security. This message stores the value of the factor into rawSplitFactor as of the ex-date provided. The new value of adjustmentFactor is then computed and stored on this date as well. For example, if XYZ has a 2-for-1 split on April 15, 1997 the procedure to add this factor is:
Named Security XYZ addSplitFactorOf: 2 onExDate: 970415 asDateRaw split information is normally updated during the nightly production cycle. It should never be necessary to use this message directly; however, it can be used to update splits that have not been included in the daily updates or to correct problems during your session. If you insert new dates into the time series, you should also execute the message rebuildAdjustmentFactor to recompute all the adjustment factors for the security.
Prices
The message price returns the split-adjusted, currency-adjusted price for a specific security. The price is accessed as of the current date by default. The value is split adjusted through the current date and is in the base currency of the security by default. For example:
Named Security XYZ price
returns XYZ's latest price value in XYZ's base currency,
Named Security XYZ :price asOf: 9702 .or
9702 evaluate: [ Named Security XYZ price ]returns the latest price as of Feb 1997 in the base currency , and
"CAN" asCurrency evaluate: [ Named Security XYZ price ]returns the latest price in Canadian dollars. All values are split-adjusted through the current date. The message _price returns the unadjusted price as of the current evaluation date.
The message priceChangeFrom:to: returns the price percentage change value between two dates. For example:
Named Security XYZ priceChangeFrom: 970201 to: 970331returns the percentage change in price between these two dates.
The actual time series of prices and volumes is stored in the time series property pricingSeries. The values in this time series are instances of the class PriceRecord, which has been created to store the information needed to fully describe the prices and volume for a date. In addition to the price and volume values, instances of this class maintain the currency and date of the price so that proper split and currency adjustments can happen. To access a full price record for a security, use the message getPriceRecord. For example, the expression:
Named Security XYZ getPriceRecord highreturns the latest high price for security XYZ.
The following messages are defined for the class PriceRecord:
| Message | Description |
|---|---|
| _price | actual closing price value provided |
| price | split/currency adjusted closing price value |
| _high | actual high value provided |
| high | split/currency adjusted high value |
| _low | actual low value provided |
| low | split/currency adjusted low value |
| _open | actual open value provided |
| open | split/currency adjusted open value |
| _volume | actual volume value provided |
| volume | split adjusted volume value |
| recordDate (date) | date of price record |
| adjustmentDate | date through which stored price was adjusted (default is recordDate) |
| adjustmentFactor | split adjustment factor based on adjustmentDate |
| currencyFactor | currency adjustment relative to price date |
| baseCurrency | actual currency of unadjusted price |
| currency | current currency (security's by default) |
| security | actual security instance |
For example, to display the latest adjusted and unadjusted prices and volumes XYZ, use:
Named Security XYZ getPriceRecord
do: [ recordDate print: 12 ;
price print ; #-- adjusted value
_price print ; #-- unadjusted value
volume print ; #-- adjusted value
_volume print ; #-- unadjusted value
adjustmentFactor print ;
currencyFactor print ;
newLine print ;
] ;
To access a full price record as of a specific date, use
the expression:
Named Security XYZ :getPriceRecord asOf: 9711 .
do: [ price print ; _price print ] ;
or
9711 evaluate:
[ Named Security XYZ getPriceRecord
do: [ price print ; _price print ] ;
] ;
Both return the full price record as of Nov 30, 1997 from which you
can access the adjusted and unadjusted prices.
The method prices has been defined to return the full time series of prices. For example, to display the adjusted and unadjusted prices and volumes for all points in XYZ's time series in 1997, you could use:
Named Security XYZ prices from: 970101 to: 9712 .
do: [ ^date print: 12 ; #-- date in time series
recordDate print: 12 ; #-- date in record - should match
price print ; #-- adjusted value
_price print ; #-- unadjusted value
volume print ; #-- adjusted value
_volume print ; #-- unadjusted value
adjustmentFactor print ;
currencyFactor print ;
newLine print ;
] ;
Note that the prices message returns an actual time series
object so you need not prepend the ":" to the message name.
Prices and volumes are normally updated as part of the daily cycle. Permanent corrections to any incorrect data are also made using a database administration procedure. If you need to temporarily fix a value for your working session, you can use the PriceFeed class.
Updating Prices
Normally you will use the PriceFeed class to update your pricing data. This section describes some of the "behind-the-scenes" protocol that supports the pricing structures. Note that this material is for the truly curious and is not needed in order to use or update pricing data.
To efficiently track daily pricing information for a large universe of securities, a special set of pricing structures have been defined. These structures organize pricing into separate objects spaces by year. Within a year, a separate physical price store is created for each quarter. In addition, a separate object space and store are used to track end-of-month prices so that month-end prices are physically clustered near one another.
Vision databases that include the Portfolio Management Application Layer are created with a separate object space to hold prices for eom, the last 5 years, the current year, and the next two years. The script InstallPriceSpaces.inv, located in the /localvision/bootstrap directory, was used to set up these spaces.
To create a space for a new year, execute and save the following code without specifying an object space:
PriceTools setupYear: YYYYMMDD ; Utility updateNetworkWithAnnotation: "Price Space Setup" ;This will create a new object space to hold pricing data associated with the year implied by the full date supplied as a parameter. You need to run this for each year separately.
In addition to creating a new object space, the setupYear: message creates four new PriceRecord stores, one for each quarter in the year. Each PriceRecord store holds daily prices for one quarter. The individual PriceRecord stores can be accessed via the time series property dailyStoreXRef defined at PriceTools. The dates in this time series correspond to the first date of the quarter associated with the store. To determine the number of instances in each store use:
PriceTools :dailyStoreXRef
do: [ "The store beginning on " print ; ^date print ;
" contains " print ; ^self instanceList count print ;
" PriceRecord instances and is stored in object space" print ;
^self objectSpace printNL ;
] ;
A separate space and price store have been created to hold all
EOM price records. Since the number of month-end records is
significantly smaller than the number of daily records, it is
often sufficient to create a single price store for all eom price
records. This store is created as part of the initial database.
You can create additional EOM stores if desired by executing and saving the following code without specifying an object space:
PriceTools setupEOMFrom: YYYYMMDD ; Utility updateNetworkWithAnnotation: "EOM Price Space Setup" ;The individual eom PriceRecord stores can be accessed via the time series property eomStoreXRef defined at PriceTools. The dates in this time series correspond to the first date for which eom records are collected in this store. To determine the number of instances in each eom store use:
PriceTools :eomStoreXRef
do: [ "The store beginning on " print ; ^date print ;
" contains " print ; ^self instanceList count print ;
" PriceRecord instances and is stored in object space" print ;
^self objectSpace printNL ;
] ;
Prices are connected to securities using a two-level time series
structure. The time series property pricingSeries is defined
at Security to track the pricing data associated with a security
by year. The time points in pricingSeries represent the
start of each year for which the security has pricing data. The
value as each point is a time series of the daily price records
for the security for the year. In other words, the expression:
Named Security IBM :pricingSeries asOf: 96 .returns a time series whose dates represent days in 1996 and whose values represents IBM's PriceRecord instance for the associated date. To retrieve and display all price records for IBM in 1996, you can use the expression:
Named Security IBM :pricingSeries asOf: 96 . do: [ ^date print: 15 ; price printNL ] ;The method prices has been defined to collapse this mutli-level time series into a single flat time series containing all pricing records for the security. For example:
Named Security IBM prices from: 960101 to: 961231 . do: [ ^date print: 15 ; price printNL ] ;In general, you do not need to have knowledge of this two-level time series structure. Unless you are explicitly writing methods that need to make use of the two-level time series structure, you should use the prices message to access your data.
A number of messages have been defined to encapsulate the price update process. The message getOrCreateRecordIn: security asOf: date has been defined at the class PriceTools to create and modify PriceRecord information. The method will create a new PriceRecord for the supplied security/date combination if it does not exist and returns the existing or newly created PriceRecord which can be updated using one or more of the following messages in succession:
PriceTools getOrCreateRecordIn: Named Security IBM asOf: 971015 .
setPriceTo: number .
setVolumeTo: number .
setHighTo: number .
setLowTo: number .
setOpenTo: number .
setBidTo: number .
setAskTo: number .
setAdjustmentDateTo: date .
setBaseCurrencyTo: currency .
;
If you are creating the PriceRecord, you will minimally want to set
the price (i.e., the closing price) and the base currency. By
default, the adjustment date is the same as the record date. If
the pricing data is adjusted through a date that is different from
the date of the price, you should set the adjustment date to this date.
The volume, high, low, open, bid, and ask values can be set or
modified as appropriate. The base currency applies to all data
values except for volume. The adjustment date is used to access
all data values.
A basic methodology for updating prices is outlined below:
Security defineMethod: [ | loadPricesFromFile: file |
!recs <- file asFileContents asLines
extendBy:
[ !fields <- ^self breakOn: ", " ;
] .
select: [ fields count >= 2 ] .
extendBy:
[ !sec <- fields at: 1 . stripBoundingBlanks ;
!security <- ^global Named Security uniformAt: sec ;
!price <- fields at: 2 . asNumber ;
!date <- fields at: 3 . asNumber asInteger asDate else: ^date ;
!curr <- fields at: 4 . stripBoundingBlanks ;
!currency <- ^global Named Currency at: curr .
else: ^global Named Currency USD ;
] ;
recs select: [ security isntNA && price > 0 ] .
do: [ ^global PriceTools getOrCreateRecordIn: security asOf: date .
setPriceTo: price .
setBaseCurrencyTo: currency ;
] ;
] ;
This approach is fine for small pricing universes and for updates
that process a single day of prices at a time. If you are planning
to update multiple days of prices within a single feed and you are
storing information for a large universe of daily securities, you
will want to include an additional step that correctly allocates
some underlying pricing components. This modification is outlined
below:
Security defineMethod: [ | loadPricesFromFile: file |
!recs <- file asFileContents asLines
extendBy:
[ !fields <- ^self breakOn: ", " ;
] .
select: [ fields count >= 2 ] .
extendBy:
[ !sec <- fields at: 1 . stripBoundingBlanks ;
!security <- ^global Named Security uniformAt: sec ;
!price <- fields at: 2 . asNumber ;
!date <- fields at: 3 . asNumber asInteger asDate else: ^date ;
!curr <- fields at: 4 . stripBoundingBlanks ;
!currency <- ^global Named Currency at: curr .
else: ^global Named Currency USD ;
] ;
!valid <- recs select: [ security isntNA && price > 0 ] ;
#-- preallocate dates for all valid records
valid groupedBy: [ security ] .
do: [ !dateList <- groupList send: [ date ] ;
^global PriceTools
preallocateDatesIn: dateList forSecurity: ^self ;
] ;
#-- process valid records
valid
do: [ ^global PriceTools getOrCreateRecordIn: security asOf: date .
setPriceTo: price .
setBaseCurrencyTo: currency ;
] ;
] ;
Dividends
The message dividend returns the split-adjusted, currency-adjusted dividend value for a specific security. The dividend value is accessed as of the current date by default. The data is adjusted through the current date in the base currency of the security by default. For example:
Named Security XYZ dividendreturns XYZ's latest dividend value in US dollars,
Named Security XYZ :dividend asOf: 9702 .or
9702 evaluate: [ Named Security XYZ dividend ]returns XYZ's latest dividend value as of Feb 1997 in US dollars, and
"CAN" asCurrency evaluate: [ Named Security XYZ dividend ]returns XYZ's latest dividend value in Canadian dollars. All values are split-adjusted through the current date. The message getDivsFrom:to: returns the total dividends paid by the security between the two dates supplied. For example:
Named Security XYZ getDivsFrom: 970201 to: 970331returns the total dividends paid between the two dates. The message totalReturnFrom:to: returns the total return between the two dates supplied. For example, to access the total return for the same period use:
Named Security XYZ totalReturnFrom: 970201 to: 970331The actual time series of dividend values is stored in the time series property dividendSeries. The values in this time series are instances of the class DivRecord, which has been created to store the information needed to fully describe a dividend. In addition to the dividend value, instances of this class maintain the currency and date of the dividend so that proper split and currency adjustments can happen.
The following key messages are defined for the class DivRecord:
| Message | Description |
|---|---|
| _div | actual dividend value provided |
| div | fully adjusted dividend value |
| recordDate (date) | dividend date |
| adjustmentDate | date through which stored dividend was adjusted (default is recordDate) |
| adjustmentFactor | split adjustment factor based on adjustmentDate |
| currencyFactor | currency adjustment relative to price date |
| baseCurrency | actual currency of unadjusted dividend |
| currency | current currency (security's by default) |
| security | actual security instance |
To support the possibility of future reorganization of this data structure for performance and storage reasons, the method dividends has been defined to return this time series. Unless you are explicitly writing methods that update the time series, you should use the dividends message to access your data. In this way, your code will be immune to any changes that may be made to the underlying data structure that manages the actual time series.
For example, to display the adjusted and unadjusted dividend values for each of XYZ's dividends, you could use:
Named Security XYZ dividends
do: [ ^date print: 12 ; #-- date in time series
date print: 12 ; #-- date in record - should match
div print ; #-- adjusted value
_div print ; #-- unadjusted value
adjustmentFactor print ;
currencyFactor print ;
newLine print ;
] ;
Note that the dividends message returns an actual time series
object so you need not prepend the ":" to the message name.
Dividends are normally updated as part of the daily cycle. Permanent corrections to any incorrect dividends are also made using a database administration procedure. If you need to temporarily fix a value for your working session, you can use the DivFeed class.
Security and Company Identification