Vision Portfolio Management Application: Sample Production Environment

Overview of the Sample Production Environment

Sample jobs that can be used in a typical portfolio management environment are provided with all installations that include the Portfolio Management Application Layer. The following sample files can be found in the /localvision/production/Updates/jobs directory: MiscMasters.job, SecurityPlus.job, HoldingsPlus.job, HoldingsPlus.okay, PricePlus.job, and DailyCleanup.okay. For many installations, these jobs can be used as is. Alternatively, they can be modified to address the specific needs of your production environment.

Daily Processing Requirements

Each day, new security, portfolio, pricing, and holdings information is available. Data may be available throughout the evening from a combination of internal and external sources. Daily production feeds in a portfolio management environment can be grouped as follows:

The miscellaneous master files should be processed first, followed by the security and related files. The daily cleanup should be delayed until holdings and pricing have been updated.

Assumptions

To use the supplied jobs as is, you will need to create extract files with the following names:

  • AggAccountMaster.yyyymmdd
  • AggMembers.yyyymmdd
  • CompanyMaster.yyyymmdd
  • CountryMaster.yyyymmdd
  • CurrencyExchange.yyyymmdd
  • CurrencyMaster.yyyymmdd
  • Holdings.yyyymmdd
  • HoldingsEom.yyyymmdd
  • IndustryMaster.yyyymmdd
  • PortfolioMaster.yyyymmdd
  • Prices.yyyymmdd
  • SecTypeMaster.yyyymmdd
  • SectorMaster.yyyymmdd
  • SecurityAliases.yyyymmdd
  • SecurityMaster.yyyymmdd
  • Splits.yyyymmdd
The file extension yyyymmdd is the "as of" date of the data. This is normally the prior end-of-day date. Data files containing data extracted from the appropriate internal and external systems should be placed in the /localvision/upload/feeds directory.

When the current day's miscellaneous master files are available, the update job MiscMasters should be scheduled. The contents of the token file should be the date used as a file extension for these files. For example:

    cd /localvision/production/status
echo 19981115 > pending/MiscMasters
This job should begin within five minutes. When completed, the pending token will automatically be removed and the token Done.MiscMasters or Error.MiscMasters will be placed in the /localvision/production/status/tokens directory.

When the current day's security and related files are available, the update job SecurityPlus should be scheduled. The contents of the token file should be the date used as a file extension for these files. For example:

    cd /localvision/production/status
echo 19981115 > pending/SecurityPlus
This job should begin within five minutes. When completed, the pending token will automatically be removed and the token Done.SecurityPlus or Error.SecurityPlus will be placed in the /localvision/production/status/tokens directory. Note that you should not schedule this job to run prior to scheduling the MiscMasters job for the day.

When the current day's account and holdings files are available, the update job HoldingsPlus should be scheduled. The contents of the token file should be the date used as a file extension for these files. For example:

    cd /localvision/production/status
echo 19981115 > pending/HoldingsPlus
This job should begin within five minutes. When completed, the pending token will automatically be removed and the token Done.HoldingsPlus or Error.HoldingsPlus will be placed in the /localvision/production/status/tokens directory. Note that you should not schedule this job to run prior to scheduling the SecurityPlus job for the day.

When the current day's price and related files are available, the update job PricePlus should be scheduled. The contents of the token file should be the date used as a file extension for these files. For example:

    cd /localvision/production/status
echo 19981115 > pending/PricePlus
This job should begin within five minutes. When completed, the pending token will automatically be removed and the token Done.PricePlus or Error.PricePlus will be placed in the /localvision/production/status/tokens directory. Note that you should not schedule this job to run prior to scheduling the SecurityPlus job for the day.

A Typical Daily Cycle

  1. Each weekday night, cron starts a new production cycle. Once a set of daily maintenance tasks have completed successfully, the update and report processing daemons are started. These processes will wait for update and report jobs to be scheduled in the /localvision/production/status/pending directory. If the pre-conditions for a pending job are met, the job will run. The pending token will be removed and the appropriate tokens and output will be placed in the status/tokens, status/logs, and/or status/reports directory. When multiple jobs are pending, they are run in the order that they were scheduled.

  2. Throughout the evening, one or more processes are run to extract data from various internal and external sources. Since requirements and tools vary from installation to installation, you will need to create the jobs to extract this information as needed. Extracted data files should be placed in the directory /localvision/upload/feeds using the names listed above. When the current day's extracts are available, the files MiscMasters, SecurityPlus, HoldingsPlus, and PricePlus should be created in the status/pending directory. These files should contain the date used as the file extension for the extracted files. These jobs do not need to be scheduled at the same time; however, they should be scheduled in this order.

  3. To see which jobs are active and which are still pending, use:
      cd /localvision/production/status
ls active pending

    To see the status of completed jobs use: 

      cd /localvision/production/status
ls tokens

    To view a log file use:

      cd /localvision/production/status
more logs/HoldingsPlus.log

The MiscMasters Job

The sample job MiscMasters.job is located in the directory /localvision/production/Updates/jobs. It will update the Vision database with the following files, if they exist in the /localvision/upload/feeds directory:

File Name Feed Name
CurrencyMaster.yyyymmdd CurrencyMaster
CountryMaster.yyyymmdd CountryMaster
SectorMaster.yyyymmdd SectorMaster
IndustryMaster.yyyymmdd IndustryMaster
SecTypeMaster.yyyymmdd SecurityTypeMaster

This job executes the following steps:

  1. Get the file extension to use for the current data files from the pending token.
  2. Create the log file MiscMasters.yyyymmdd in the directory /localvision/production/Updates/workArea/Internal. If the log already exists, append it to the end of a running log for that date and start a new log.
  3. Write job header to log file.
  4. Start a vision session logging output to the log file.
  5. Load the CurrencyMaster, CountryMaster, SectorMaster, IndustryMaster, and SecurityTypeMaster data.
  6. Commit the new data to the database.
  7. Check the log to make sure the update succeeded.
  8. Copy the log to the /localvision/production/status/logs directory.
  9. Move the data files into the /localvision/upload/feeds/done directory.
  10. Return 0 if the job succeeded, 1 if there was an error.

The SecurityPlus Job

The sample job SecurityPlus.job is located in the directory /localvision/production/Updates/jobs. It will update the Vision database with the following files, if they exist in the /localvision/upload/feeds directory:

File Name Feed Name
SecTypeMaster.yyyymmdd SecurityTypeMaster
SecurityAliases.yyyymmdd SecurityAliases
CompanyMaster.yyyymmdd CompanyMaster
SecurityMaster.yyyymmdd SecurityMaster

This job executes the following steps:

  1. Get the file extension to use for the current data files from the pending token.
  2. Create the log file SecurityPlus.yyyymmdd in the directory /localvision/production/Updates/workArea/Internal. If the log already exists, append it to the end of a running log for that date and start a new log.
  3. Write job header to log file.
  4. Start a vision session logging output to the log file.
  5. Load the SecurityTypeMaster, SecurityAliases, CompanyMaster, and SecurityMaster data.
  6. Commit the new data to the database.
  7. Check the log to make sure the update succeeded.
  8. Copy the log to the /localvision/production/status/logs directory.
  9. Move the data files into the /localvision/upload/feeds/done directory.
  10. Return 0 if the job succeeded, 1 if there was an error.

The HoldingsPlus Job

The sample job HoldingsPlus.job is located in the directory /localvision/production/Updates/jobs. It will update the Vision database with the following files, if they exist in the /localvision/upload/feeds directory:

File Name Feed Name
PortfolioMaster.yyyymmdd PortfolioMaster
AggAccountMaster.yyyymmdd AggAccountMaster
AggMembers.yyyymmdd PortfolioAggregates
Holdings.yyyymmdd HoldingsFeed
HoldingsEom.yyyymmdd HoldingsFeed

This job executes the following steps:

  1. Get the file extension to use for the current data files from the pending token.
  2. Create the log file HoldingsPlus.yyyymmdd in the directory /localvision/production/Updates/workArea/Internal. If the log already exists, append it to the end of a running log for that date and start a new log.
  3. Write job header to log file.
  4. Start a vision session logging output to the log file.
  5. Load the PortfolioMaster, AggAccountMaster, PortfolioAggregates, and HoldingsFeed data. Two holdings extracts are normally provided. The file Holdings.yyyymmdd contains the latest holdings and the file HoldingsEom.yyyymmdd contains a replacement for the latest end-of-month holdings.
  6. Commit the new data to the database.
  7. Check the log to make sure the update succeeded.
  8. Copy the log to the /localvision/production/status/logs directory.
  9. Move the data files into the /localvision/upload/feeds/done directory.
  10. Return 0 if the job succeeded, 1 if there was an error.

The PricePlus Job

The sample job PricePlus.job is located in the directory /localvision/production/Updates/jobs. It will update the Vision database with the following files, if they exist in the /localvision/upload/feeds directory:

File Name Feed Name
Prices.yyyymmdd PriceFeed
Splits.yyyymmdd SplitsFeed
Divs.yyyymmdd DivFeed
CurrencyMaster.yyyymmdd CurrencyMaster
CurrencyExchange.yyyymmdd ExchangeRateFeed

This job executes the following steps:

  1. Get the file extension to use for the current data files from the pending token.
  2. Create the log file PricePlus.yyyymmdd in the directory /localvision/production/Updates/workArea/Internal. If the log already exists, append it to the end of a running log for that date and start a new log.
  3. Write job header to log file.
  4. Start a vision session logging output to the log file.
  5. Load the PriceFeed, SplitsFeed, DivFeed, CurrencyMaster, and ExchangeRateFeed data.
  6. Commit the new data to the database.
  7. Check the log to make sure the update succeeded.
  8. Copy the log to the /localvision/production/status/logs directory.
  9. Move the data files into the /localvision/upload/feeds/done directory.
  10. Return 0 if the job succeeded, 1 if there was an error.

Related Topics