Background processing is a powerful feature that allows RefTracker defined batch processes to be run automatically at defined intervals.

Background processes defined in RefTracker are run by a web service called ReftService that is set up at the time of installation of your RefTracker system.  You can check that the Background process function is running for your system by looking at the information about “ReftService last checked for processes to run” located above the tabs in any Background processing screen, or you can see a log of the start time for each run under System>Utilities>Information>View System log and then selecting to view the Background processing log.

You manage what that process will do by going to System>Batch process menu>Background processing and then the particular background process that you want to set up or change. By default, Supervisors and System administrators can enable and disable these processes, and change the frequency with which they run.

If the Background processing menu is not showing in your system, you need to turn it on at parameter 5.2 Allow background processing. This parameter has been provided so that ALL background processes can be stopped or restarted, rather than having to stop them individually. This parameter also allows in-house systems to choose whether background processing should be allowed by the system. It is not essential for background processing to be running in your system. Of course if it is not running, none of the background processes like automatic email importing will occur, but they can all be run manually. Housekeeping must be done on all systems of course, but if it has not been done by the background process within the last four hours, it will be done, each time a system administrator logs on.

Some background processes (such as the reporting processes) will not show in the list of Background processes, unless a process has been set up to use that background process. The process will be automatically created, and appear in the menu once a task that uses it has been created.

There are a number of functions that ensure ReftService can run securely, that presume that ReftService is running on the same server as RefTracker. Parameter 9.5 facilitates situations where that is not the case.

The background processing Overview screen

Because there are a lot of background processes, the background process Overview screen (at System>Batch process menu>Background processing>Overview) shows when each process last ran in the order in which they ran, how often they run, when they last ran and will run next, and a summary of what they did in that last run.

Importantly if you need to find out more about any of these processes, or make a change to the process including disabling it, you can just click on the Process name and you’ll be taken to the Overview tab for that specific background process where the status, parameters and schedule can be reviewed and changed, and a log of the runs for the specific process can be seen.  Click here for more information about the background process Overview screen.

Using the background processing screen – aspects common to all processes

All of the background processes are provided by RefTracker and work in a consistent manner, however each process will provide different parameters and allow different aspects to be changed by your organisation. The following information about how to use the Background processing screen uses the Import email process as an example but is relevant to all background processes.

You can move between processes and instances of processes by selecting the process you want from the drop down in the top right of the screen. If there is more than one instance of that process the right hand box on that line will become a drop down list that allows you to choose between the instances. For example, if you have more than one email account defined and enabled in Email importing, RefTracker will have created an instance of the Import email process for each of those email accounts so that importing from each account can be separately scheduled.

You can also easily move to an overview of all last runs of processes using the [System overview] link.  To left of that link will be a link to show just the last run of each process of the same type (group) as the process you are currently viewing (e.g. the last run of all email importing processes).

You can save changes to the process and manually run them using the buttons at the bottom of each screen. The buttons that show will vary according to the Level of the process being viewed and can be any of the following:

• Update – save any changes made to this screen (such as changes to the processes Status). 
• Delete – delete this instance of this user defined process. 
• Run process now – run this process now. 
• Add – create another instance of this user defined process.

Four tabs display for each process:

• Overview provides a summary of what the process does, how it has been configured, when it was last run, and provides the opportunity to turn it on or off.
• Parameters allows you to customise how the process will run for you.
• Schedule is where you set how frequently the process will run.
• Process log provides a summary of all times that this process has been run over the last period defined by parameter 0.5.

To set up a process you must review all of the settings in the Overview, Parameters and Schedule tabs, so let’s look at these tabs in detail:

The Overview tab:

For user defined processes, the key attribute on this page is whether the process is enabled or disabled.

Process type: RefTracker will make available process types. Currently there are only two process types – Housekeeping and Email import, but in the future there will be more. A process type does a particular function, and instances of the process type (see Process name below) will do variations on that process.

Level: System mandatory indicates a function that is required by RefTracker and cannot be changed by you (such as Housekeeping). System optional indicates a process that is required by RefTracker and can be configured to meet your specific needs in relation to whether it is in use and how frequently it runs. Import email is an example of this level of process. User defined indicates a process that is optional and configurable in all respects. There are not yet any user defined processes.

Process name: This is the name of the specific instance of this process type and can be set by you for user defined processes, but is set by RefTracker for system processes. A process type can be used multiple times allowing variations on that process to be run, for example, if you have more than one email account defined in Email importing, RefTracker will automatically create a separate process of the type Import email in order to be able to bring in emails from each of those different email addresses. The Process name allows multiple instances of the Process type to be individually identified.

Descriptive note: This is a short description/explanation of what this instance of this process does. You provide this description for user defined processes and RefTracker provides it for system processes.

Status: This is where you define whether a process is to be automatically scheduled or not. System mandatory processes are always set to Enabled. Enabled means that this process will be run automatically according to the timing specified in the Schedule tab. Disabled means that this process is not currently being automatically run. (it can still be run on a once off manual basis using the “Execute now” button).

Average process time: This is the average amount of time in hours:minutes:second:milliseconds that this process has taken to run over the runs summarised in the Process log tab.

Process will next run: This is the time at which this process is next scheduled to run. Note that the next execution is calculated by adding the Frequency period specified in the Schedule tab to the last execution date and time. It is an approximation of the time at which the process will be started, that can be effected by the frequency of the scheduled task and the other tasks scheduled to run at the same time.

Summary of last processing run: This section of the Overview tabs summarises the information that has been placed in the Process log tab about the last time this process was executed (if it has been run within the last parameter 0.5 period of time.

The Parameters tab:

Use this tab to set the schedule for automatic processing of this instance of this process.

Start date: the date and time from which this process should begin running automatically.

End date or Does not expire: the date and time at which this instance of this process should stop running, or an indication that it should continue running indefinitely (tick “Does not expire”).

Frequency/Interval: Select how frequently this instance of this process should run. Other parameters will appear according to the frequency that you choose.  The process will run with that frequency but not every run will have data to report, so, for example, if you are expecting a file to be emailed to you hourly, and there is no output that hour, you will not receive an email (but you will see a note that the process ran, with nothing to report, in the Process log).

If you choose “Minutely” you will also need to choose a minute “Interval” e.g. 2 – will mean that the process will run every 2 minutes (usually within a minute ) with the minutes being counted from when that process last ran.

If you choose “Hourly” the process will run x hours after it last ran where you select x hours apart.  A time is not specified so the Hourly process does not run exactly x hours apart, but gets triggers to run the next time batch processing runs, after x hours has expired from the last time it ran (which will result in it appearing to creep forward a little each time it runs). 

If you choose “Daily” you can specify the time of day that the process should run using the “Start at” parameter that will display.  In fact, with this and all of the following interval options, the process will run the next time the background process runs after that selected time – usually within a minute.

Choosing “Weekly” allows you to choose the day (including more than one day) of each week, and time, at which the process should run.

Choosing “Monthly (day)” allows you to choose the day of each month e.g. first Saturday, and time, at which the process should run.

Choosing “Monthly (date)” allows you to choose the date in each month that the process should run e.g. the 1st of the month, or 2nd, or 3rd, etc up to the 28th, and also the last day of the month.

Choosing “Quarterly” allows you to choose the date in each quarter that the process should run e.g. the 1st day of the quarter, or 2nd, or 3rd, etc up to the 28th, and also the last day of the quarter.

Choosing “Annual (cal year)” allows you to choose the date in each calendar year that the process should run e.g. the 1st day of the calendar year, or 2nd, or 3rd, etc up to the 28th, and also the last day of the calendar year.

Choosing “Annual (fin year)” allows you to choose the date in each financial year (with first month of the year specified by parameter 2.19) that the process should run e.g. the 1st day of the financial year, or 2nd, or 3rd, etc up to the 28th, and also the last day of the financial year.

The minimum amount of time between runs is 1 minute, and at each run the server looks for processes that have exceeded their specified frequency to run. 

Please take the overall processor load into account and only run processes as frequently as necessary, and delete them when they are no longer required. If the processes due to run take more than one minute, the system will not look for more processes to run until the next minute deadline arrives after the run finishes – so too many processes running every minute will result in all processes running, at best, every two minutes.

Start at: The time at which the process will run.  It only displays if you have chosen a frequency of daily, weekly or monthly.

Process will next run: The Frequency you select will be used to calculate the next time that this instance of the process will run by adding that information to the date and time that the instance of this process last ran. This means that if you choose to run every Monday and it last ran at 9:23pm it will run at 9:23pm on Monday. This process ensures that response times are not effected by processes being schedule to run on the hour. Where appropriate, Process parameters will allow selection of the start and end dates for data to be included so that processes can be run for specific date and time ranges without having to be run exactly at those times.

When you have reviewed the Overview, Parameters and Schedule tabs and the process is configured to meet your needs, you are ready to have the process start running automatically. Set Status in the Overview tab to “Enabled” to run automatically, or you can test the process using the “Run process now” button  on any tab without first enabling the process (the last execution summary in the overview tab will be updated to show the results of that process run).

Whenever the process is run (automatically or manually), an entry for that run is created in the Process log tab.

The Process log tab:

Every time a process runs an entry is recorded in the process log, ant they are kept for at least the periods selected at parameters 0.5 and 0.6.
The records for processes that took no action are deleted after 24 hours.

Other process records are retained according to how often that process is scheduled:

Minutely – the last 1200 runs

Hourly – the last 800 runs

Daily – the last 100 runs

Weekly – the last100 runs

Monthly – the last 50 runs

Quarterly – the last 25 runs

Annually – the last 10 runs

The information presented in the Process log tab is as follows:

[System overview]: The System overview link provides an easy way to see the last runs of this and other all other processes.  To the left of it is another Overview link that will show you just the last runs of the other processes of this same type (e.g. all email importing processes).

ReftService last check: This show how long since the background process last check to see if this process needed to be run.  This is particularly useful information if you have the “Exclude idle records ” box ticked as it confirms that the process is running even if it is not actually processing anything.

Exclude idle records: When looking at the background processing logs, such as the email importing log, it is often handy to be able to see the results of every processing run (such as when waiting for an email to be imported).  However, on other occasions it would be handy to be easily able to see the last time something was successfully imported. To easily facilitate both of these scenarios we have added an “Exclude idle records” tick box to the Process log tab display.  Just tick the “Exclude idle records” tick box to see the just the imports that did something, or untick it to see every run, even if nothing happened.  The system will remember your preference for this setting and display it that way next time you come into this screen.

And the columns in the table are:

• Start time: date and time that the process instance started running. 
• End time: date and time that the process instance finished running. 
• Status: “Success” indicates the import process instance ran successfully. “Idle” means there was no email to import on that occasion. “Unknown” means the process was running at the time that this screen was displayed. Redisplaying after the process has finished running will show the status of the completed process. 
• Summary: Summary of the functions performed by the process. This summary will contain different information for each instance. A log entry is created every time a task is run, however only information about runs that actually processed something will be retained after 24 hours.
  No action means there was nothing for the process to do on this occasion.
  No results means that the process was run and it returned nothing (so, for example, if it was a report that was run with no results, no email of results will have been sent).
  Failed mean that the scheduled run errored out.
• Staff: The name of the staff member who ran this process if it was run manually using “Execute now”, or blank if run automatically by Batch processing. 
• Summary details e.g. Log or Mail log: some processes will provide more detailed logs of the processing performed. Text in this column is hyperlinked to a display of the more detailed log – in the case of the Import email process the link takes you to the same Email importing log summary page as is provided for the process under Systems>Batch process menu> Data import/export>Email importing where the details of the email account being imported from was set up.