upsp.intensity_mapping.patching

clusterFiducials(fiduals_visible, rmat, tvec, cameraMatrix, distCoeffs, boundary_thickness, buffer_thickness_in)[source]

Clusters input fiducials based on image location and image size

A cluster is made of all fiducials with a path of overlap between them, i.e. A is overlapping B which is overlapping C and D, therefore A, B, C, and D are all in the same cluster. It is considered overlap if the internal pixels to one fiducial overlap the internal or boundary pixels of another fiducial. A cluster can be a single fiducial if there is no overlap.

This function clusters targets, then returns the clusters.

Parameters

  • fiduals_visible (list) – Each fiducial is a dict with (at a minimum) ‘tvec’ and ‘target_type’ attributes. The ‘tvec’ attribute gives the fiducial’s location and ‘target_type’ is a string denoting the type of fiducial (most commonly ‘dot’ or ‘kulite’)

  • rmat (numpy.ndarray, shape (3, 3), float) – Rotation matrix from camera to object

  • tvec (numpy.ndarray, shape (3, 1), float) – Translation vector from camera to object

  • cameraMatrix (numpy.ndarray, shape (3, 3), float) – The (openCV formatted) camera matrix for the camera

  • distCoeffs (numpy.ndarray, shape (5, 1) or (5,), float) – The (openCV formatted) distortion coefficients for the camera

  • boundary_thickness (int) – Thickness of boundary (in pixels)

  • buffer_thickness_in (float) – Buffer (in inches) to add to fiducials when determining internals applied radially (increases effective radius of fiducial by buffer_thickness_in)

Returns

clusters – List of clusters, where each cluster is a list of fiducials from fiducials_visible.

Return type

list

get_cluster_internal_and_boundary(cluster, rmat, tvec, cameraMatrix, distCoeffs, boundary_thickness, buffer_thickness_in)[source]

Returns a list of internal and boundary pixels for the input cluster

An internal pixel is either directly a part of the fiducials or in between fiducials. A boundary pixel is any pixel within buffer pixels of an internal pixel and is not an internal itself

Parameters

  • cluster (list) – List of fiducials in the cluster. Each fiducial is a dict with (at a minimum) ‘tvec’ and ‘target_type’ attributes. The ‘tvec’ attribute gives the fiducial’s location and ‘target_type’ is a string denoting the type of fiducial (most commonly ‘dot’ or ‘kulite’)

  • rmat (numpy.ndarray, shape (3, 3), float) – Rotation matrix from camera to object

  • tvec (numpy.ndarray, shape (3, 1), float) – Translation vector from camera to object

  • cameraMatrix (numpy.ndarray, shape (3x3), float) – The (openCV formatted) camera matrix for the camera

  • distCoeffs (numpy.ndarray, shape (5, 1) or (5,), float) – The (openCV formatted) distortion coefficients for the camera

  • boundary_thickness (int) – thickness of boundary (in pixels)

  • buffer_thickness_in (float) – Buffer (in inches) to add to fiducials when determining internals applied radially (increases effective radius of fiducial by buffer_thickness_in)

Returns

  • internals (numpy.ndarray, shape (n, 2)) – Positions (x, y) of the n internal points

  • bounds (numpy.ndarray, shape (m, 2)) – Positions (x, y) of the m boundary pixels

See also

get_fiducial_boundary_map_from_internal_map

Determines boundary pixels from bit mask of internal pixels

get_fiducial_pixel_properties

Returns the pixel properties of the input fiducial

get_fiducial_boundary_map_from_internal_map(internal_map, boundary_thickness)[source]

Determines boundary pixels from bit mask of internal pixels

internal_map is a bitwise mask (image) where 1 is an internal and 0 is not. internal_map needs to be big enough to contain all boundary pixels. That means it needs to have a buffer of 0’s around it that is boundary_thickness thick on all sides (a number of columns of 0’s left of leftmost internal equal to boundary_thickness. Same for rightmost. And similarly rows above and below)

Performs n dilation operations on the internal_map with a 3x3 square kernel where n is equal to boundary_thickness. boundary_map (return value) is the result minus internal_map

Parameters

  • internal_map (numpy.ndarray) – Bitwise mask of internal pixels

  • boundary_thickness (int) – thickness of boundary (in pixels)

Returns

boundary_map – Bitmask of boundary pixels. Same shape as internal_map input

Return type

numpy.ndarray

See also

get_fiducial_internal_and_boundary

Return internal and boundary pixel positions for the input fiducial

get_cluster_internal_and_boundary

Returns a list of internal and boundary pixels for the input cluster

get_fiducial_internal_and_boundary(tgt, rmat, tvec, cameraMatrix, distCoeffs, boundary_thickness, buffer_thickness_in)[source]

Return internal and boundary pixel positions for the input fiducial

An internal pixel a part of the fiducials (minimum axis aligned bounding rectangle). A boundary pixel is any pixel within buffer pixels of an internal pixel and is not an internal itself

Parameters

  • tgt (dict) – dict with (at a minimum) ‘tvec’ and ‘target_type’ attributes. The ‘tvec’ attribute gives the fiducial’s location and ‘target_type’ is a string denoting the type of fiducial (most commonly ‘dot’ or ‘kulite’)

  • rmat (numpy.ndarray, shape (3, 3), float) – Rotation matrix from camera to object

  • tvec (numpy.ndarray, shape (3, 1), float) – Translation vector from camera to object

  • cameraMatrix (numpy.ndarray, shape (3x3), float) – The (openCV formatted) camera matrix for the camera

  • distCoeffs (numpy.ndarray, shape (5, 1) or (5,), float) – The (openCV formatted) distortion coefficients for the camera

  • boundary_thickness (int) – thickness of boundary (in pixels)

  • buffer_thickness_in (float) – Buffer (in inches) to add to fiducials when determining internals applied radially (increases effective radius of fiducial by buffer_thickness_in)

Returns

  • internals (numpy.ndarray, shape (n, 2)) – Positions (x, y) of the n internal points

  • bounds (numpy.ndarray, shape (m, 2)) – Positions (x, y) of the m boundary pixels

See also

get_fiducial_boundary_map_from_internal_map

Determines boundary pixels from bit mask of internal pixels

get_fiducial_pixel_properties

Returns the pixel properties of the input fiducial

get_fiducial_pixel_properties(tgt, rmat, tvec, cameraMatrix, distCoeffs, boundary_thickness, buffer_thickness_in)[source]

Returns the pixel properties of the input fiducial

Pixel properties refers to projected location, pixel size (adjusted for focal length, distance to camera, and diameter), and minimum axis-aligned bounding box

Parameters

  • tgt (dict) – dict with (at a minimum) ‘tvec’ and ‘target_type’ attributes. The ‘tvec’ attribute gives the fiducial’s location and ‘target_type’ is a string denoting the type of fiducial (most commonly ‘dot’ or ‘kulite’)

  • rmat (numpy.ndarray, shape (3, 3), float) – Rotation matrix from camera to object

  • tvec (numpy.ndarray, shape (3, 1), float) – Translation vector from camera to object

  • cameraMatrix (numpy.ndarray, shape (3, 3), float) – The (openCV formatted) camera matrix for the camera

  • distCoeffs (numpy.ndarray, shape (5, 1) or (5,), float) – The (openCV formatted) distortion coefficients for the camera

  • boundary_thickness (int) – thickness of boundary (in pixels)

  • buffer_thickness_in (float) – Buffer (in inches) to add to fiducials when determining internals applied radially (increases effective radius of fiducial by buffer_thickness_in)

Returns

  • tgt_proj (dict) – Fiducial projection which is a dict with keys ‘target_type’, and ‘proj’ which map to a string and list of positions (length 2, (x, y)) respectively. ‘target_type’ matches the input ‘target_type’.

  • targ_size_px (float) – Fiducial size in pixels accounting for the focal length, distance to camera, and diameter. Does not take model geometry into account, and is either exactly accurate or an over estimate (likely a mild overestimate).

  • t_min (numpy.ndarray, shape (2,), int) – Upper left corner of the minimum axis aligned bounding box of the fiducial plus boundary pixels.

  • t_max (numpy.ndarray, shape (2,), int) – Bottom right corner for the bounding box.

See also

get_fiducial_internal_and_boundary

Return internal and boundary pixel positions for the input fiducial

get_cluster_internal_and_boundary

Returns a list of internal and boundary pixels for the input cluster

get_target_node_idxs(nodes, tgts, buffer_thickness_in)[source]

Returns the indices of all nodes that are inside of targets

Returns list of indices of nodes that fall within the patch of a target. This is calculated by finding the distance from every node to every target. All nodes that are a distance less than tgt['size'] from the target are marked as ‘invalid’ and the indices of those nodes are returned

This is calculated in a two step procedure for a speedup. It is based on the fact that Euclidean distance <= Manhattan distance <= sqrt(3) * euclidean distance, and that Manhattan distance is much faster to calculate than Euclidean distance since there is no square or square root operation.

Step 1 calculates the Manhattan distance (L1 norm) from every node to every target. If the Manhattan distance is greater than sqrt(3) * tgt['size'], we know for certain that the node is at least tgt['size'] units of Euclidean distance from the target. This will be the vast majority of nodes (all but ~300 for example launch vehicle w/ ~1 million nodes).

Step 2 checks the Euclidean distance of each node that failed the Manhattan distance check. Nodes that fall within tgt['size'] of any target are marked as invalid and their index is returned.

If a large portion of the nodes are near a target, this process would be slow since it is effectively double calculating any node near a target. However, since the targets are relatively small compared to the surface area of the model, it results in a roughly 2X speedup.

Parameters

  • nodes (numpy.ndarray, shape (N, 3), float) – A numpy array of the X, Y, and Z values of the nodes

  • tgts (list) – Each target is a dictionary with (at a minimum) a ‘tvec’ and ‘size’ attribute. The ‘tvec’ attribute gives the target’s location and the ‘size’ gives the Euclidean distance from the center of the target to the perimeter

  • buffer_thickness_in (float) – Buffer (in inches) to add to fiducials when determining internals applied radially (increases effective radius of fiducial by buffer_thickness_in)

Returns

nodes_in_targets – Sorted array of the indices (np.int32) of the nodes that are inside of a target

Return type

numpy.ndarray

patchFiducials(fiduals_visible, inp_img, rmat, tvec, cameraMatrix, distCoeffs, boundary_thickness, buffer_thickness_in)[source]

Patches clusters in the inp_img

Parameters

  • fiduals_visible (list) – Each fiducial is a dict with (at a minimum) ‘tvec’ and ‘target_type’ attributes. The ‘tvec’ attribute gives the fiducial’s location and ‘target_type’ is a string denoting the type of fiducial (most commonly ‘dot’ or ‘kulite’)

  • inp_img (numpy.ndarray, np.uint8) – Input image to be patched. rmat and tvec should be aligned to image

  • rmat (numpy.ndarray, shape (3, 3), float) – Rotation matrix from camera to object

  • tvec (numpy.ndarray, shape (3, 1), float) – Translation vector from camera to object

  • cameraMatrix (numpy.ndarray, shape (3x3), float) – The (openCV formatted) camera matrix for the camera

  • distCoeffs (numpy.ndarray, shape (5, 1) or (5,), float) – The (openCV formatted) distortion coefficients for the camera

  • boundary_thickness (int) – Thickness of boundary (in pixels)

  • buffer_thickness_in (float) – Buffer (in inches) to add to fiducials when determining internals applied radially (increases effective radius of fiducial by buffer_thickness_in)

Returns

out_img – Image with patched fiducials

Return type

numpy.ndarray, float

See also

get_fiducial_internal_and_boundary

Return internal and boundary pixel positions for the input fiducial

get_cluster_internal_and_boundary

Returns a list of internal and boundary pixels for the input cluster

polyfit2D

Finds the polynomial fit using the boundary pixels

polyval2D

Finds the value for the internal pixels using the polynomial fit

polyfit2D(bounds, Is)[source]

Finds the polynomial fit using the boundary pixels

Parameters

  • bounds (numpy.ndarray, shape (n, 2) of floats) – (x, y) position of boundary pixels in local coordinates (i.e. leftmost boundary pixel has x coordinate of 0. Topmost has coordinate of 0). bounds[n] corresponds to Is[n]

  • Is (numpy.ndarray, shape (n,) of floats) – Intensity value of boundary pixels pixels. Is[n] corresponds to bounds[n]

Returns

coeffs – Polynomial fit coefficients

Return type

numpy.ndarray, shape (n, 1)

See also

polyval2D

Finds the value for the internal pixels using the polynomial fit

polyval2D(internals, coeffs)[source]

Finds the value for the internal pixels using the polynomial fit

Parameters

  • internals (numpy.ndarray, shape (n, 2) of floats) – (x, y) position of internal pixels in local coordinates (i.e. leftmost boundary pixel has x coordinate of 0. Topmost has y coordinate of 0)

  • coeffs (numpy.ndarray, shape (n,)) – Polynomial fit coefficients

Returns

Is – Estimated intensity for internals

Return type

numpy.ndarray, shape (n,)

See also

polyfit2D

Finds the polynomial fit using the boundary pixels