torchsky.catalogs¶

Unified sky crossmatch entry point.

Supports: - crossmatch(cat_left, cat_right, radius_arcsec, ...) for :class:Catalog objects - crossmatch(left_lon, left_lat, right_lon, right_lat, radius_arcsec=..., ...) for coordinate tensors - crossmatch((left_lon, left_lat), (right_lon, right_lat), radius_arcsec, ...) - crossmatch(left_arrow, right_arrow, radius_arcsec, streamed=False, ...) for Arrow tables/paths

Chunked exact angular crossmatch on one in-memory tile.

Matchers (choose one radius rule):

• sky: radius_arcsec (scalar or per-left-row), pair kept when separation <= radius.
• skyerr: left_err_arcsec / right_err_arcsec (scalar or per row, either side optional, missing side = 0), pair kept when separation <= err_left[i] + err_right[j]. Candidate windows use max(err_left) + max(err_right); the exact test uses each pair's sum.
• sky + distance (STILTS sky with a 1-d cut): also pass left_distance, right_distance and distance_tol; a pair must additionally satisfy |d_left - d_right| <= distance_tol (NaN never matches).

find selects which pairs are returned:

• "best" / "best1": nearest right row per left row (right rows may repeat); ties go to the lowest right index.
• "best2": nearest left row per right row (left rows may repeat); ties go to the lowest left index.
• "mutual": STILTS find=best. Exactly the mutual-best pairs (best1 intersected with best2, same tie rules); a row whose best partner prefers another row is left unmatched.
• "symmetric": greedy one-to-one (not STILTS). Ascending separation, ties by lowest left then lowest right index; every row is used at most once and the assignment is maximal.
• "all": every pair inside the threshold.

"all" is ordered by left index, separation, then right index; the other modes by left index, then right index. Use :func:~torchsky.catalogs.join_rows for join semantics with unmatched rows.

radius_arcsec may be a positive scalar or a per-left-row tensor. Pass right_index to reuse a nested HealpixIndex (find best1/all and sky matcher only). device moves inputs onto that device when given. On MPS the work dtype is float32, so thresholds below the float32 position resolution are not meaningful there.

Parameters:

Returns:

Examples:

>>> import torch
>>> from torchsky.catalogs import crossmatch_sky
>>> result = crossmatch_sky(torch.tensor([0.0]), torch.tensor([0.0]), torch.tensor([0.0]), torch.tensor([0.0]), radius_arcsec=1.0)
>>> result.right_index.tolist()
[0]

Crossmatch Arrow inputs through temporary HEALPix tiles.

Both catalogs are streamed once (partitioned by coarse HEALPix pixel into temporary Parquet files with a coarse_pixel column). Per-pixel tiles are then loaded one at a time for fine-grained distance computation. Coordinate memory scales with the largest tile and its neighboring right tiles. Pixel metadata and returned pairs remain in RAM. Indices always refer to the original input row order, including across batches. Use them to select native ID columns without converting IDs to float.

For uniformly distributed sky positions and a well-chosen coarse nside, per-pixel tile size is typically a few thousand rows regardless of whether the total catalog has 10^6 or 10^9 rows.

Parameters¶

left_source, right_source Each can be a pyarrow.Table, pyarrow.ipc.RecordBatchReader, or a path string / pathlib.Path to an Arrow IPC file (Feather format v2). File sources use memory-mapped I/O. radius_arcsec Match radius in arcseconds. left_columns, right_columns (ra_column_name, dec_column_name) for each catalog. batch_size Row count per streaming batch during the tiling phase. find "best" selects one nearest right row per left row, breaking ties by original right row index. "all" returns every pair in radius.

Streamed crossmatch where the left catalog arrives in batches.

The right catalog must fit in memory (required for the coarse HEALPix index). The left catalog is consumed incrementally from left_batches, which yields (ra_deg, dec_deg) tensor pairs.

Parameters¶

left_batches Iterator yielding (ra_tensor, dec_tensor) pairs (each 1-D float64). Batches are concatenated in order and their indices are mapped back into the final left_index output. right_ra_deg, right_dec_deg Full right-catalog coordinates (1-D float64). radius_arcsec Match radius in arcseconds. right_chunk_size Right-side chunk size for the inner distance loop.

Pixels whose centers lie in the cap of radius radius_rad.

Small caps expand nested neighbours (O(hops^2)). Wide caps use query_disc. Callers that need a sound match covering pass r + 2ρ.

Coarsest power-of-two NSIDE whose max pixel radius is <= radius_rad.

If the cap is larger than an nside=1 pixel, return 1. If even max_nside pixels are larger than the cap, return max_nside (covering stays sound via the r + 2ρ pad; tiles are just coarser).

Equal-key join (STILTS exact): O((N + M) log M + K).

Keys are 1-d tensors (integers or floats; NaN never matches). Separations are returned as zeros. find follows :func:crossmatch_sky; every pair scores equally, so ties resolve by lowest row index ("symmetric" pairs the lowest rows of each key first).

Match C >= 2 catalogues (STILTS tmatchn).

Each catalogue is a Catalog or a (lon_deg, lat_deg) pair. A pair of rows matches when separation <= radius_arcsec or, with err_arcsec (one scalar/tensor/None per catalogue), <= err_a + err_b.

mode="pairs" (multimode=pairs): every other catalogue is matched to catalogs[reference] with find="best1": each reference row takes its nearest partner (ties by lowest row) in every other catalogue, so one entry may serve several reference rows. Output rows are all reference rows in order (-1 where no partner), then, for each other catalogue in order, its rows that are no reference row's partner (alone in their row). Two non-reference rows are never linked directly.

mode="group" (multimode=group): friends-of-friends over all cross-catalogue pairs; one output row per group, ordered by the group's smallest stacked row (catalogue order, then row). A group may hold several rows of one catalogue (chains); the representative of catalogue c is the member with the smallest separation to any linked member of another catalogue, ties by lowest row index. Same-catalogue pairs are not links.

Pairs within one catalogue (STILTS tmatch1).

Returns every pair i < j (no self pairs, each pair once) with separation <= radius_arcsec or, with err_arcsec (scalar or per row), separation <= err[i] + err[j]. With group=True also returns friends-of-friends groups (connected components of the pair graph) as per-row group_id and group_size (STILTS action=identify). Candidate generation is the declination-strip index, so cost is O(N log N + K); components use scatter-min union-find.

Row-index pairs for a STILTS join type, -1 where a side is absent.

join is one of 1and2 (matched pairs), 1or2 (matched pairs plus every unmatched row of both), all1 / all2 (matched pairs plus the unmatched rows of table 1 / 2), 1not2 / 2not1 (unmatched rows only). Output order: matched pairs as given, then unmatched left rows ascending, then unmatched right rows ascending. Costs O(K + N + M).

Returns: