upsp.cam_cal_utils.visibility

class VisibilityChecker(grid_path, oblique_angle=70, epsilon=0.0001, debug=False, debug_nogrid=False)[source]

Bases: object

Visibilty object class for occlusion and viewing angle checking

This class creates an object for checking visibility of a list of points with associated normals. The intended use is for the external calibration to check target visibility, and to check visibility of the the model mesh nodes for node-pixel mapping.

For the visibility check, it is assumes that the entire model is within the field of view of the camera. This simplifies the check for 2 reasons:

  1. We don’t need to check if a node is within the field of view

  2. We don’t need to check if an intersection is behind the camera

    • A ray is drawn from a given point to the camera. If that ray intersects a node, it is deemed not visible. However, if there is a node behind the camera that intersects with the ray, this will still be seen as an intersection and flagged as not visible. Even though the intersection behind the camera does not truly occlude the given point.

Parameters

  • grid_path (str) – Filepath to the grid file

  • oblique_angle (float, optional) – Maximum allowable oblique viewing angle. Viewing angle for a node pointed directly at the camera and in the center of the field of view of the camera is 0 degrees

  • epsilon (float, optional) – Intersection tolerance. For the ray tracing intersection check, the origin of the ray is offset from the model by a distance of epsilon in the direction of the normal

  • debug (bool, optional) – Debug parameter to speed up grid loading. It is suggested to use this debug for all development, and to leave it False for all non-development case. If True, looks for '{{filename}}_primitives.npy' in ‘.cache’, where filename is the filename from grid_path. If it finds it, loads from the .npy file rather than reading and processing the grid file. If it doesn’t find it, it will read and process the grid file as normal, and save the numpy array as '{{filename}}_primitives.npy' in ‘.cache’

  • debug_nogrid (bool, optional) – Debug parameter to speed up computation. It is suggested to use this debug for development if occlusions are not needed for development work. There is no reason to leave this parameter for any non-development work. If True, instead of loading a real grid file, it loads a fake one with a single, extremely small primitive (effectively having ‘no grid’). This makes BVH operations much faster.

angle_between(v1, v2)[source]

Returns the angle in radians between vectors v1 and v2

Parameters
Returns

angle – Angle (in radians) between v1 and v2

Return type

float

See also

unit_vector

Returns the unit vector of the vector

is_back_facing

This is the ‘slow’ version of the back face culling check

does_intersect(origin, direction, return_pos=False)[source]

Function that determines if a point is occluded by the object mesh

Creates a ray from origin with given direction. Checks for intersection of ray with the BVH

Parameters
Returns

result – True if node is occluded and False if node is not occluded

Return type

bool

get_faces_and_face_normals()[source]

Returns the faces and face normals of all grid nodes

Returns

  • faces (numpy.ndarray, shape (n, 3, 3)) – Each face is a (3, 3) with (i, 3) corresponding to a node, and each node having an (x, y, z). Faces are ordered counter-clockwise.

  • face_normals (numpy.ndarray, shape (n, 3)) – The face normal is (x, y, z). face_normals[i] corresponds to face[i]

get_tvecs_and_norms()[source]

Returns the tvecs and tvec normals of all grid nodes

Returns

  • tvecs (numpy.ndarray, shape (n, 3)) – Each tvec is corresponds to the (x, y, z) of a node.

  • norms (numpy.ndarray, shape (n, 3)) – Each norm is an (x, y, z) of the node’s normal vector norms[i] corresponds to tvecs[i]. The norm is the normal of the first face to contain the tvec. Each tvec can appear in multiple faces, so the first is used. In practice, each face that contains the node will have a different normal vector, but in general the difference is a relatively small angle.

is_back_facing(t, n)[source]

This is the ‘slow’ version of the back face culling check

This function is mostly for legacy/regression purposes

TODO: This does not filter points on the horizon of the model (~90 degrees oblique viewing angle). It does seem to filter those above 70 degrees (might not though, very little testing was done). It might have something to do with the clip in angle between

Parameters
Returns

back_facing – True if the node is back facing. False if it is not back facing

Return type

bool

See also

angle_between

Returns the angle in radians between vectors ‘v1’ and ‘v2’

is_back_facing

This is the ‘slow’ version of the back face culling check

is_back_facing_fast_vectorized

Returns array of booleans for which nodes are back facing

is_back_facing_fast(t, n)[source]

Returns True if the node is back facing, otherwise returns False

This re-write is significantly faster than the naive approach, but not as fast as the vectorized approach. This function is mostly for legacy/regression purposes

The node is back facing if angle between vectors < oblique angle

Mathematically this is done as:

\[\arccos\left(\frac{t \cdot n}{\|t\| \|n\|}\right) < \textrm{oblique angle}\]

Since arccos is an expensive operation, re-write as:

\[\frac{t \cdot n}{\|t\| \|n\|} < \cos(\textrm{oblique angle})\]

Since division is more expensive than multiplication, re-write as:

\[t \cdot n < \|t\| \|n\| \cos(\textrm{oblique angle})\]

Using \(\|a\|^2 = a \cdot a\), we get:

\[(t \cdot n) (|t \cdot n|) < (t \cdot t) (n \cdot n) \cos^2(\textrm{oblique angle})\]

Where the absolute value is required on the LHS to preserve sign. All elements of RHS are positive since t and n contain only real numbers

\(\cos^2(\textrm{oblique angle})\) is calculated and cached when the oblique angle changes, giving an additional speedup

Parameters
Returns

back_facing – True if input is back facing (viewing angle > maximum oblique angle). False if input is not back facing (viewing angle <= maximum oblique angle)

Return type

bool

See also

is_back_facing

This is the ‘slow’ version of the back face culling check

is_back_facing_fast_vectorized

Returns array of booleans for which nodes are back facing

is_back_facing_fast_vectorized(t, n)[source]

Returns array of booleans for which nodes are back facing

See is_back_facing_fast() for explaination of math. To vectorize \(a \cdot b\) we use np.sum(a*b, axis=1)

Parameters
Returns

back_facing – output[i] is True if node[i] is backfacing (viewing angle > maximum oblique angle). output[i] is False if node[i] is not backfacing (viewing angle <= maximum oblique angle).

Return type

numpy.ndarray, bool

See also

is_back_facing

This is the ‘slow’ version of the back face culling check

is_back_facing_fast

Returns True if the node is back facing, otherwise returns False

is_visible(tvec_model_to_camera, nodes, normals, return_angles=False)[source]

Returns list of nodes that are visible

Currently only checks for oblique viewing angle and occlusion, assumes all nodes are within FOV of camera

Parameters

  • tvec_model_to_camera (numpy.ndarray, shape (3, 1), float) – translation vector from model to camera

  • nodes (numpy.ndarray, shape (N, 3), float) – X, Y, Z position of nodes to be checked. nodes[i] is associated with normals[i]

  • normals (numpy.ndarray, shape (N, 3), float) – Normal vectors. normals[i] is associated with nodes[i]

  • return_angles (bool, optional) – If True, return angles for each visible node as well.

Returns

  • visible (numpy.ndarray) – Numpy array of the indices of the nodes that are visible

  • angles (numpy.ndarray) – Angles of visible nodes. Only returned when return_angles=True

See also

is_back_facing_fast_vectorized

Returns array of booleans for which nodes are back facing

does_intersect

Function that determines if a point is occluded by the object mesh

load_mesh(grid_path)[source]

Loads the grid file into vertices and indices

Parameters

grid_path (string) – Filepath to the grid file

Returns

t – Dict with keys “vertices” and “indices”. t["vertices"] is a (N, 3) array of the model vertices. t["indices"] is a (N, 3) array of ints where each row refers to a model face i. The vertices that make up face i are t["vertices"][n], where n is (3, 1) from t["indices"][i]

Return type

dict

package_primitives(t)[source]

Packages the primitives of the grid file into the BVH format

Converts the vertices and indices into a list of primitives in the form:

[t0p0x, t0p0y, t0p0z, t0p1x, t0p1y, t0p1z, t0p2x, t0p2y, t0p2z, ...]
Parameters

t (dict) – Dict with keys “vertices” and “indices” from load_mesh().

Returns

primitives – Packaged primitives

Return type

numpy.ndarray

tri_normal(face)[source]

Returns the normal of a face with vertices ordered counter-clockwise

Parameters

face (numpy.ndarray, shape (3, 3), float) – X, Y, Z positions of the face vertices. face[i] contains x, y, z of vertex i

Returns

normal – Normal vector of input face

Return type

numpy.ndarray, shape (3,)

unit_vector(vector)[source]

Returns the unit vector of the vector

Helper function for angle_between()

Parameters

vector (numpy.ndarray, shape (n, 3) float) – Array of vectors

Returns

unit_vector – Unit vector(s) corresponding to vector

Return type

numpy.ndarray

See also

angle_between

Returns the angle in radians between vectors ‘v1’ and ‘v2’

update_oblique_angle(oblique_angle)[source]

Updates object elements related to the oblique viewing angle

Parameters

oblique_angle (float) – Maximum allowable oblique viewing angle. Viewing angle for a node pointed directly at the camera and in the center of the field of view of the camera is 0 degrees