How to Create a Data Request IOR

1 Important Background Information

1.1 What is a Data Request IOR?

A Data Request IOR includes sets of instructions for STIX to process its observations into certain data products (e.g. L1 or L4 Spectrogram) and send them back to Earth. These instructions correspond to data requests made by the community or the STIX team. (To learn more, see What is a Data Request?) Data Request IORs can only be issued during STIX Analysis Windows which are scheduled and defined on SOOP KITCHEN.

1.2 Data Request IOR Criteria

A Data Request IOR must be compiled in line with the following criteria:

• Highest priority data requests must be included first. Data Requests can have three priorities: High, Medium, and Low.
• The total number to telecommands in a single 24 hour period must be less than 300. This includes any other IORs that send commands on the same day. However, this limit is not always a hard one. Sometimes ESA does accept slightly more than 300 telecommands in a single day.
• The total data volume of the requests must be within the telemetry budget defined for the STIX Analysis Window. This will be recorded on SOOP Kitchen for the given analysis window.

While the telemetry budget should not be exceeded, current algorithms tend to over-estimate the data volume of data request IORs. Moreover, it is important not to waste telemetry as we can not utilized unused telemetry budget afterwards. It is therefore important to use as much of the budget as possible.

1.3 Data Request Status

Data Requests have one of three statuses:

• Executed: The requested data has been received from STIX by the STIX Data Center and is available.
• Processed: The data request has been included in an IOR on the STIX Data Center server, but the data has not yet received. This could be because the IOR has not been sent to STIX. Or the IOR has been sent but the data has not yet been received by the STIX Data Center server.
• Pending: The data request has been successfully saved to the STIX Data Center server, but has not yet been included in an IOR by the STIX operators.

1.4 Relationship to Other IORs

While it is possible to mix data requests with other telecommands, it is standard prcatice to create an IOR just for data requests and another for other operations telecommands. An exception might be to include non-data request telecommands that MUST be run during the STIX analysis window. This makes it easier to ensure that STIX is operating nominally during the STIX analysis window and the telecommand limit of 300/day is being used more efficiently. Therefore, IORs should not overlap in time.

2 Generate a Data Request IOR

2.1 Required Software

See Required Software.

2.2 How to Generate a Data Request IOR

2.2.1 Retrieving the Data Requests

• Confirm no other IORs are sending commands during the STIX analysis window.
• Export the data requests from the STIX Data Center as a JSON file. There are two API options, URL or website:
  • URL API
    • Enter the following URL https://pub023.cs.technik.fhnw.ch/request/user-data-requests/ior/json/<start_id>/<end_id> where <start_id> and <end_id> are the start and end data request IDs to include, e.g. https://pub023.cs.technik.fhnw.ch/request/user-data-requests/ior/json/1030/1400. Only data requests in the range whose status is "Pending" will be included.
    • Save the JSON file to your local computer.
  • Website API
    • Go to Expert User section of the STIX website. On the left side bar click "Operations" followed by "Data Request Manager". (Or go directly by clicking here.) This gives a table of all data requests.
    • Click the "Load pending requests" button. Unless there is a specific reason, use the default values in the pop-up window and click "OK". This filters the list to only pending data requests (i.e. those not already in an IOR that has been uploaded to the STIX server).
    • Access a hidden toolbar by double-clicking the word "requests" in "STIX user data requests" above the "New" button.
    • Check the tickboxes for the data requests to be included. Alternatively, click "AutoSel" to automatically select <300 data requests.
      • Data requests should be selected in order of highest priority.
      • Within each priority level, the requests involving older data should be picked first. Note requests for older data do not always correspond to lower IDs.
      • The table can be sorted according to a certain parameter by clicking on the arrows next to the column name. Depending on how many requests there are, this may take a while. The order can be reversed by clicking the same arrows again.
      • Note that aspect data requests are not selected by "AutoSel" because the website software currently can't determine whether they have been processed or executed. Their status is always "Pending". So the operator must know whether specific aspect data requests should be included.
      • The number of data requests is limited to <300 by the daily telecommand limit of 300. Although each data request is one telecommand, additional telecommands are needed to faciliate the data requests which are automatically generated when the data requests are exports as a JSON. See below.
    • Ensure the total data volume (QL estimated) is equal to the telemetry budget. This may require manually including/excluding some larger data requests. But be mindful of the 300 telecommand limit.
      • Data requests can be manually added/removed by checking/unchecking the tick boxes.
      • The telemetry can be found in SOOPKitchen by hovering your cursor of the STIX Analysis window.
    • "Export" the list by clicking "Export". This will save the list as a JSON. Save it in the location you are using to build the IOR.

2.2.2 Building the IOR

• It is typical to build an IOR specially for the data requests. If possible, other telecommands required during the STP can be put in another IOR. This IOR should be executed on a different day to the data request IOR so as not to waste the daily 300 telecommand limit. However, data request and other telecommands can be put into the same IOR if needed.
• Open STIX-Starlet.
• Prepare the IOR up until the point of adding sequences/telecommand as outlined in How To Generate an IOR.
• Load in the data request JSON file by clicking yellow "open json" button at the right end of the toolbar.
  • The JSON file will group data requests based on common detector and pixel masks. The first two commands of the group will set the detector and pixel masks, respectively. Subsequent commands will correspond to data requests.
• To process data requests, STIX must be in Configuration or Nominal mode. Confirm that STIX will be in of these modes when the STIX analysis window starts. This is usually determined by the IOR sent at the start of the STP which power cycles the instrument. If it isn't, add a command at the start of the IOR changing the mode.
• Edit the wait times
  • The default wait time between each group is +8 hrs. This needs to be changed for the first group to 2 seconds.
  • Wait times for subsequent groups must give enough time for all requests in the previous group to be downloaded. Estimating this time is difficult. It is safest to leave the wait time as +8hrs (default). If you have several groups and/or if the STIX Analysis window is short, this may need to be reduced.
  • Delays between L4 spectrogram requests aren't important. They will be queued. These can be left at 10 seconds.
  • If including an aspect data request, allow 30 minutes for each request in the wait time.
• Once all data request wait times have been edited, convert them to absolute times clicking "Request" -> "Delta to Absolute"
• Click Load JSON
• Click Generate
• Once ready, Submit IOR for Internal Review in the usual way.
• Once approved by the STIX team, Submit IOR to SOC in the usual way.