101.1. Alert retrieval service#
101.1. Alert retrieval service¶
For the Rubin Science Platform at data.lsst.cloud.
Data Release: Prompt Products
Container Size: Large
LSST Science Pipelines version: r29.2.0
Last verified to run: 2026-06-17
Repository: github.com/lsst/tutorial-notebooks
DOI: 10.11578/rubin/dc.20250909.20
Learning objective: How to obtain single alert packets from the RSP's alert retrieval service.
LSST data products: Alert packets.
Packages: lsst.rsp.RSPClient, lsst.rsp.get_service_url, fastavro.
Credit: Originally developed by Rubin Data Management team members. Please consider acknowledging them if this notebook is used for the preparation of journal articles, software releases, or other notebooks.
Get Support: Everyone is encouraged to ask questions or raise issues in the Support Category of the Rubin Community Forum. Rubin staff will respond to all questions posted there.
1. Introduction¶
The RSP's alert retrieval service enables the contents of single alert packets to be retrieved by alert ID number, in one of three formats: Avro, JSON, or FITS. See also the technical note "Design of an Alert Retrieval Service in the RSP" (SQR-114).
Alert packets are files of measurements for sources detected in difference images that are streamed to brokers within a few minutes of image acquisition. Alerts are a Prompt Product, and are the result of Rubin's Difference Image Analysis (DIA) and Alert Production (AP) pipelines.
Brokers are the recommended way to do time-domain science with alerts because their functionality includes, e.g., filtering, cross-match, and classification, and users can explore and retrieve scientifically-relevant subsets via their web-based user interfaces or API clients.
Why use the RSP's alert retrieval service?
The alert retrieval service is useful when only the contents of a single alert packet are desired, and the alert ID is known.
Otherwise, brokers are the best way to do real-time science with the alerts, and scientific analyses on longer timescales (days to weeks) can use the other Prompt Products (queryable databases, images).
Additional resources:
- Alert Production database schema.
- Prompt Products documentation.
- Learn more about the LSST alert brokers.
This tutorial focuses on how the alert retrieval service works, and options for packet retrieval formats (e.g., FITS, JSON, schema-only, cutouts-only).
Related tutorials: The 200-level tutorial on alert packets focuses on the details of the packets' schema and data contents, and how packets for static-sky and moving objects differ.
1.1. Import packages¶
Import the fastavro package for reading alerts in the default Avro format. Import the base64 package for reading image stamps in the JSON format.
Import the RSPClient and get_service_url in order to access the alert retrieval service.
Also import a range of other plotting and analyis packages.
import fastavro
import base64
import matplotlib.pyplot as plt
import numpy as np
import pandas as pd
import io
from astropy.io import fits
from astropy.nddata import CCDData
from astropy.visualization import (MinMaxInterval, LinearStretch,
ImageNormalize)
from lsst.rsp import RSPClient, get_service_url
from lsst.utils.plotting import (get_multiband_plot_colors,
get_multiband_plot_symbols)
1.2. Define parameters and functions¶
Establish the connection to the alert retrieval service by getting the url for the service and instantiating the RSPClient.
url = get_service_url("alerts", "prompt")
client = RSPClient("")
Use the recommended colorblind-friendly palette to represent Rubin filters.
filter_colors = get_multiband_plot_colors()
filter_names = list(filter_colors.keys())
filter_symbols = get_multiband_plot_symbols()
At the time this tutorial was prepared, the ALeRCE broker had identified the object associated with this alert as a potential Active Galactic Nucleus (AGN).
Note that the alert retrieval service will also accept IAU-formatted IDs in the form of: LSST-AP-DS-170059317401616524, where the numerical portion is the diaSourceId.
alert_id = "170059317401616524"
response = await client.get(url, params={"ID": alert_id})
response.raise_for_status()
response?
Type: Response String form: <Response [200 OK]> File: /opt/lsst/software/stack/conda/envs/lsst-scipipe-10.1.0-exact/lib/python3.12/site-packages/httpx/__init__.py Docstring: <no docstring>
Read the packet with the fastavro.reader. Get the schema that is embedded in the packet. Print only the keys, with an option to display the full schema.
packet = fastavro.reader(io.BytesIO(response.content))
schema = packet.writer_schema
print(f"Schema keys: {schema.keys()}")
# schema
Schema keys: dict_keys(['type', 'name', 'fields'])
Get the packet record and print the keys.
records = list(packet)
record = records[0]
print(f"Record keys: {record.keys()}")
Record keys: dict_keys(['diaSourceId', 'observation_reason', 'target_name', 'diaSource', 'prvDiaSources', 'prvDiaForcedSources', 'diaObject', 'ssSource', 'mpc_orbits', 'cutoutDifference', 'cutoutScience', 'cutoutTemplate'])
Print the values for the first three keys.
print(f" diaSourceId {record['diaSourceId']}")
print(f" observation_reason {record['observation_reason']}")
print(f" target_name {record['target_name']}")
diaSourceId 170059317401616524 observation_reason alert_cosmos target_name ddf_cosmos, lowdust
2.1.1. Triggering source¶
For the triggering diaSource record, print the band (filter), MJD (date and time at the midpoint of the exposure), and the difference-image flux and its error in nJy.
diaSource = record['diaSource']
print(f" Band {diaSource['band']}")
print(f" MJD {diaSource['midpointMjdTai']}")
print(f" Flux {diaSource['psfFlux']}")
print(f" Error {diaSource['psfFluxErr']}")
Band i MJD 61097.322050417104 Flux -3602.8203125 Error 313.1190185546875
2.1.2. Lightcurve¶
Extract the band, MJD, and PSF difference-image fluxes from the alert record as numpy arrays, and plot the lightcurve.
Begin by extracting the diaSource and previous diaSource (prvDiaSources) measurements to pandas dataframes, then concatenate them into a single dataframe.
df = pd.DataFrame(record['diaSource'], index=[0])
df_prv = pd.DataFrame(record['prvDiaSources'])
df_all = pd.concat([df, df_prv], ignore_index=True)
fig = plt.figure(figsize=(6, 3))
for f, filt in enumerate(filter_names):
df_pick_band = df_all.loc[df_all['band'] == filt]
if len(df_pick_band) > 0:
plt.plot(df_pick_band['midpointMjdTai'], df_pick_band['psfFlux'], filter_symbols[filt], ms=5, mew=1,
alpha=0.7, mec=filter_colors[filt], color='None', label=filt)
plt.xlabel('MJD')
plt.ylabel('diaSource PSF flux [nJy]')
plt.legend(bbox_to_anchor=(1.05, 1), handletextpad=0, loc='upper left')
plt.tight_layout()
plt.show()
Figure 1: The PSF difference-image flux lightcurve.
2.1.3. Image stamps¶
Extract and display the three image stamps (cutouts) from the packet record.
stamps = [record['cutoutScience'],
record['cutoutTemplate'],
record['cutoutDifference']]
stamp_names = ['Science', 'Template', 'Difference']
fig, ax = plt.subplots(1, 3, figsize=(9, 3))
for s, (stamp, name) in enumerate(zip(stamps, stamp_names)):
hdul = fits.HDUList.fromstring(stamp)
data = hdul[0].data
image = CCDData(data.astype("float32"), unit='nJy')
norm = ImageNormalize(image, interval=MinMaxInterval(),
stretch=LinearStretch())
plt.sca(ax[s])
plt.imshow(image, origin='lower', norm=norm, cmap='gray')
plt.colorbar()
ax[s].set_title(name)
plt.tight_layout()
plt.show()
Figure 2: The science, template, and difference-image stamps.
Clear all the data used above.
del response, packet, schema, records, record
del diaSource, stamps, stamp_names
2.1.4. Associated object¶
The record also contains the diaObject (for static-sky time-variable objects) or the ssSource and mpc_orbits (for moving objects in the Solar System).
2.2 Schema only¶
It is usually not necessary to retrieve the alert schema on its own, as the schema are embedded in the packet as demonstrated above.
However, if there is a need for schema only, this is how to get it.
response = await client.get(f"{url}/schema", params={"ID": alert_id})
response.raise_for_status()
<Response [200 OK]>
Avro schemas are written in JSON, so use the response.json() method to convert to a Python dict in order to explore.
schema = response.json()
Print some fields from the schema.
print(f"Schema name: {schema['name']}")
print(f"Schema namespace: {schema['namespace']}")
print(f"Top-level fields: {[f['name'] for f in schema['fields']]}")
Schema name: alert Schema namespace: lsst.v10_0 Top-level fields: ['diaSourceId', 'observation_reason', 'target_name', 'diaSource', 'prvDiaSources', 'prvDiaForcedSources', 'diaObject', 'ssSource', 'mpc_orbits', 'cutoutDifference', 'cutoutScience', 'cutoutTemplate']
Option to enumerate the fields and then extract the schema for field diaSource, containing the measurements for the detection in the difference-image that triggered the alert to be produced.
Alternatively, browse the alert schema online.
# fields = schema['fields']
# for f, field in enumerate(fields):
# print(f, field['name'])
# diaSource = fields[3]
# diaSource
# del fields, diaSource
Clear the data used above.
del response, schema
2.3. Stamps only¶
If only the image stamps (cutouts) from an alert are desired, have the alert retrieval service only return the images in the response.
response = await client.get(f"{url}/cutouts", params={"ID": alert_id})
response.raise_for_status()
<Response [200 OK]>
Display the three image stamps.
stamps = fits.open(io.BytesIO(response.content))
stamp_names = [hdu.name for hdu in stamps if hdu.name != "PRIMARY"]
fig, axes = plt.subplots(1, len(stamp_names), figsize=(3 * len(stamp_names), 3))
if len(stamps) == 1:
axes = [axes]
for ax, name in zip(axes, stamp_names):
image = CCDData(stamps[name].data.astype("float32"), unit='nJy')
norm = ImageNormalize(image, interval=MinMaxInterval(),
stretch=LinearStretch())
plt.sca(ax)
plt.imshow(image, origin='lower', norm=norm, cmap='gray')
plt.title(name)
plt.colorbar()
plt.tight_layout()
plt.show()
Figure 3: The difference-image, science, and template stamps.
Clear the data used above.
del response, stamps, stamp_names
3. JSON format¶
The JSON response is the full deserialized alert record. Request the JSON formatted response by using the RESPONSEFORMAT keyword.
response = await client.get(url, params={"ID": alert_id, "RESPONSEFORMAT": "json"})
response.raise_for_status()
<Response [200 OK]>
record = response.json()
print(f"Top-level keys: {list(record.keys())}")
Top-level keys: ['diaSourceId', 'observation_reason', 'target_name', 'diaSource', 'prvDiaSources', 'prvDiaForcedSources', 'diaObject', 'ssSource', 'mpc_orbits', 'cutoutDifference', 'cutoutScience', 'cutoutTemplate']
Print a few properties of the triggering difference-image source record, diaSource.
diaSource = record['diaSource']
print(f" Band {diaSource['band']}")
print(f" MJD {diaSource['midpointMjdTai']}")
print(f" Flux {diaSource['psfFlux']}")
print(f" Error {diaSource['psfFluxErr']}")
Band i MJD 61097.322050417104 Flux -3602.8203125 Error 313.1190185546875
Option to plot the lightcurve. The code is exactly the same as for the Avro packet.
# df = pd.DataFrame(record['diaSource'], index=[0])
# df_prv = pd.DataFrame(record['prvDiaSources'])
# df_all = pd.concat([df, df_prv], ignore_index=True)
# fig = plt.figure(figsize=(6, 3))
# for f, filt in enumerate(filter_names):
# df_pick_band = df_all.loc[df_all['band'] == filt]
# if len(df_pick_band) > 0:
# plt.plot(df_pick_band['midpointMjdTai'], df_pick_band['psfFlux'], filter_symbols[filt], ms=5, mew=1,
# alpha=0.7, mec=filter_colors[filt], color='None', label=filt)
# plt.xlabel('MJD')
# plt.ylabel('diaSource PSF flux [nJy]')
# plt.legend(bbox_to_anchor=(1.05, 1), handletextpad=0, loc='upper left')
# plt.tight_layout()
# plt.show()
Option to display the stamps. The code is very similar to that used for the Avro packets, except in the JSON formatted packet the image stamps are base64-encoded bytes. They need to be first decoded with base64.b64decode then read as bytes with io.BytesIO by fits.open. Note also that the stamps are in a different order than in the Avro packets.
# stamps = [base64.b64decode(record.get("cutoutDifference")),
# base64.b64decode(record.get("cutoutScience")),
# base64.b64decode(record.get("cutoutTemplate"))]
# stamp_names = ['Difference', 'Science', 'Template', ]
# fig, ax = plt.subplots(1, 3, figsize=(9, 3))
# for s, (stamp, name) in enumerate(zip(stamps, stamp_names)):
# hdul = fits.open(io.BytesIO(stamp))
# data = hdul[0].data
# image = CCDData(data.astype("float32"), unit='nJy')
# norm = ImageNormalize(image, interval=MinMaxInterval(),
# stretch=LinearStretch())
# plt.sca(ax[s])
# plt.imshow(image, origin='lower', norm=norm, cmap='gray')
# plt.colorbar()
# ax[s].set_title(name)
# plt.tight_layout()
# plt.show()
# del stamps, stamp_names
Clear the data used above.
del response, record, diaSource
4. FITS format¶
The FITS response is a multi-extension file with the following extensions.
PRIMARY: A header with telescope and instrument.ALERT: Data from the associated DIA (static-sky) or SS (moving) object.DIFFIM,SCIENCE, andTEMPLATE: Image stamps (cutouts).DIASOURCE: The triggering source record (row 0), plus any prior source records.FORCEDPHOT: Any prior forced-photometry measurements.SSSOURCE: Any associated solar system source records.
While these extensions are the same for all packets, the schema (columns) and data in the ALERT, DIASOURCE, and SSSOURCE extensions depend on whether the alert-triggering detection is associated with a diaObject (static-sky transient or variable) or an ssObject (moving object).
Get the alert packet in FITS format.
response = await client.get(url, params={"ID": alert_id, "RESPONSEFORMAT": "fits"})
response.raise_for_status()
<Response [200 OK]>
Use the fits package to open the content of response into a Header Data Unit List (HDUL), and print the names of each header.
hdul = fits.open(io.BytesIO(response.content))
for hdu in hdul:
print(hdu.name)
PRIMARY ALERT DIFFIM SCIENCE TEMPLATE DIASOURCE FORCEDPHOT SSSOURCE
PRIMARY
The primary extension contains only header information about the telescope and instrument, and no data. Show the primary header for the hdul.
hdu = hdul['PRIMARY']
hdu.header
SIMPLE = T / conforms to FITS standard BITPIX = 8 / array data type NAXIS = 0 / number of array dimensions EXTEND = T TELESCOP= 'Rubin Observatory' INSTRUME= 'LSSTCam '
Clean up.
del hdu
DIFFIM, SCIENCE, TEMPLATE
Option to display the image stamp "triplet" of science, template, and difference image for the hdul.
# fig, ax = plt.subplots(1, 3, figsize=(9, 3))
# for i, ext in enumerate(["SCIENCE", "TEMPLATE", "DIFFIM"]):
# image = CCDData(hdul[ext].data.astype("float32"), unit='nJy')
# norm = ImageNormalize(image, interval=MinMaxInterval(),
# stretch=LinearStretch())
# plt.sca(ax[i])
# plt.imshow(image, origin='lower', norm=norm, cmap='gray')
# plt.colorbar()
# ax[i].set_title(ext)
# plt.tight_layout()
# plt.show()
ALERT
The ALERT HDU contains data from the associated static-sky object (diaObject) or moving object (ssObject). The particular alert chosen in Section 1.2 is a static-sky variable, and the ALERT extension contains diaObject columns. Alerts for moving objects have different columns in the ALERT extension.
Option to get the ALERT extension HDU for the static-sky packet from hdul, and print the number of columns, and a few key column values.
# hdu = hdul['ALERT']
# print(len(hdu.columns))
# data = hdu.data
# columns = ["diaSourceId", "observation_reason", "target_name"]
# for col in columns:
# print(f"{col:<20} {data[col][0]}")
# del data, columns, hdu
DIASOURCE
The DIASOURCE extension contains data for the triggering diaSource (row 0) and all previous detections (the lightcurve).
Option to extract the diaSource data and plot the lightcurve.
# hdu = hdul['DIASOURCE']
# data = hdu.data
# fig = plt.figure(figsize=(6, 3))
# for f, filt in enumerate(filter_names):
# fx = np.where(data['band'] == filt)[0]
# if len(fx) > 0:
# plt.plot(data['midpointMjdTai'][fx], data['psfFlux'][fx], filter_symbols[filt], ms=5,
# mew=1, alpha=0.7, mec=filter_colors[filt], color='None', label=filt)
# del fx
# plt.xlabel('MJD')
# plt.ylabel('difference-image PSF flux [nJy]')
# plt.legend(bbox_to_anchor=(1.05, 1), handletextpad=0, loc='upper left')
# plt.tight_layout()
# plt.show()
# del data
# del hdu
FORCEDPHOT
The FORCEDPHOT extension contains forced measurement data on all previous visits at the position of the diaSource (the lightcurve). This includes measurements on the difference images and the science images.
Option to extract the diaSource data and plot the forced photometry lightcurve.
# hdu = hdul['FORCEDPHOT']
# data = hdu.data
# fig = plt.figure(figsize=(6, 3))
# for f, filt in enumerate(filter_names):
# fx = np.where(data['band'] == filt)[0]
# if len(fx) > 0:
# plt.plot(data['midpointMjdTai'][fx], data['psfFlux'][fx], filter_symbols[filt], ms=5,
# mew=1, alpha=0.7, mec=filter_colors[filt], color='None', label=filt)
# del fx
# plt.xlabel('MJD')
# plt.ylabel('forced difference-image flux [nJy]')
# plt.legend(bbox_to_anchor=(1.05, 1), handletextpad=0, loc='upper left')
# plt.tight_layout()
# plt.show()
# del data
# del hdu
SSSOURCE
The SSSOURCE extension contains the LSST-computed per-source instantaneous quantities at the time of observation (e.g., heliocentric and topocentric positions and velocities) for alerts associated with moving objects (i.e., Solar System objects).
Using an ID of a known Solar System object (SSO), retrieve the SSSOURCE information.
sso_alert_id = "170059294376985743"
sso_response = await client.get(url, params={"ID": sso_alert_id, "RESPONSEFORMAT": "fits"})
sso_response.raise_for_status()
sso_hdul = fits.open(io.BytesIO(sso_response.content))
for hdu in sso_hdul:
print(hdu.name)
PRIMARY ALERT DIFFIM SCIENCE TEMPLATE DIASOURCE FORCEDPHOT SSSOURCE
Note that the header cards are the same for all alerts. However, this alert will have data in the SSSOURCE extension. Extract that extension, and examine the ssObjectId of the alert.
hdu = sso_hdul['SSSOURCE']
sso_data = hdu.data
print(sso_data['ssObjectId'])
[21163646022202707]
Option to list the names of all the columns in SSSOURCE.
# for col in sso_hdul['SSSOURCE'].columns:
# print(col)