Augmentation
Augmentation allows you to create a plausible variation of an image. This is done by applying a specified ordered set of image processing operations with randomized settings within a specified range. For example, to plausibly mimic the different kinds of images a camera might capture of an object passing under it, MimAugment can randomly rotate, translate, and blur an image. This lets you generate the images that a camera can grab without the camera actually grabbing them.
[Image: MimAugment.png]
Augmentation can prove useful if you want to add a variety of modified images with which to perform test procedures, or if you want to increase the entries in your dataset when performing machine learning tasks, such as image classification or segmentation with the Aurora Imaging Library Classification module. Note, although you can easily integrate MimAugment operations from the Aurora Imaging Library Classification module, it is not typically necessary, as you can enable preset augmentation controls from the module itself (M_PRESET...). For more information, seeData augmentation and other data preparations.
Steps to augment an image
The following steps provide a basic methodology for augmenting an image:
- Allocate an image processing context for augmentation, using
MimAllocwithM_AUGMENTATION_CONTEXT. - Allocate an image processing buffer to hold augmentation results, using
MimAllocResultwithM_AUGMENTATION_RESULT. - Enable the image processing operations that the augmentation can perform, using
MimControlwithM_AUG_..._OP. By default, all operations are disabled. - Optionally, modify the range of values for the image processing operations that the augmentation can perform, using
MimControl. Aurora Imaging Library randomly chooses the values to use for the operations from within the specified range. - Optionally, set the priority (order) with which the augmentation can perform the image processing operations, using
MimControlwithM_PRIORITY. - Optionally, set the probability with which the augmentation can perform the image processing operations, using
MimControlwithM_PROBABILITY. - Perform the augmentation, using
MimAugment. This function performs the augmentation operation on a source image and places the results in either an image buffer or an augmentation result buffer, depending on the destination parameter setting. The resulting image buffer holds the modified version of the source image. The result buffer holds information about the modifications. - If you called
MimAugmentwith an augmentation result buffer, retrieve the required results, usingMimGetResult. Important results include the image processing operations performed, and the values used for them, for each call toMimAugment. To save a report of such results, callMimStreamwith theM_SAVE_REPORToperation. - Optionally, draw the resulting image of the augmentation, using
MimDrawwithM_DRAW_AUG_IMAGE. - If necessary, save your augmentation context, using
MimSaveorMimStream. - Free all your allocated objects, using
MimFree, unlessM_UNIQUE_IDwas specified during allocation.
The augmentation operation is usually run multiple times, since most augmentation related applications require multiple modified images. Typically, you should save the augmented images (MbufSave). For more information about how to use the augmentation operation, see the MimAugment.cpp example in Aurora Imaging Example Launcher.
Note, you can interactively configure and execute augmentation operations with Aurora Imaging CoPilot.
Priority
By default, the image processing operations (M_AUG_..._OP) that MimAugment can perform are prioritized (ordered) as follows:
- Affine operations, such as
M_AUG_ROTATION_OP. - Structure operations, such as
M_AUG_DILATION_OP. - Geometric operations, such as
M_AUG_FLIP_OP. - Intensity operations, such as
M_AUG_INTENSITY_MULTIPLY_OP. - Linear filter operations, such as
M_AUG_SMOOTH_DERICHE_OP. - Noise operations, such as
M_AUG_NOISE_GAUSSIAN_ADDITIVE_OP.
To modify this order, call MimControl with M_PRIORITY. Changing the order of the operations can alter their final effect; for example, smoothing an image before adding noise differs from adding noise before smoothing.
Probability
Each time you call MimAugment, Aurora Imaging Library randomly selects which enabled image processing operations to perform. By default, each operation has an equal chance of being randomly selected. To modify these chances, call MimControl with M_PROBABILITY.
Note, setting the probability to 100.0 ensures that Aurora Imaging Library performs the operation, while setting 0.0 ensures that Aurora Imaging Library will not perform the operation (this is the same as disabling the operation).
Randomness and seeds
To establish the randomness required by the augmentation operation, Aurora Imaging Library uses an internal random number generator (RNG).
All the random values that MimAugment uses trace back to a seed; that is, the first initialization value of the internal random number generator. By default, Aurora Imaging Library randomly generates a new seed each time you call MimAugment.
To specify an explicit initialization value for the internal random number generator, call MimControl with M_AUG_SEED_MODE set to M_RNG_INIT_VALUE, and then use M_AUG_RNG_INIT_VALUE to specify the actual value. Using the same seed multiple times lets you rerun (repeat) the same augmentation process with the same randomized settings. This seed is saved with the augmentation context.
You can also specify the seed directly when calling MimAugment, by using the SeedValue parameter. To do so, you must set M_AUG_SEED_MODE to M_USER_DEFINED_SEED. This seed is not saved with the context.
Note that, you can get the seed that MimAugment uses (regardless of how the seed was established) with the M_AUG_SEED_USED result, and then use that seed to repeat that exact augmentation operation.
Random rotation angle
When you enable the rotation operation (M_AUG_ROTATION_OP), a random rotation angle is generated to rotate the image. The random rotation angle can be represented with the following formula:
_AngleMin_ <= (_AngleRef_ + _i_ * _AngleStep_) ± (_AngleDelta_ / 2) < AngleMax, where:
- AngleMin (
M_AUG_ROTATION_OP_ANGLE_MIN) represents the minimum rotation angle. - AngleRef (
M_AUG_ROTATION_OP_ANGLE_REF) represents the reference angle. All other angles forM_AUG_ROTATION_OPwill be calculated relative to this angle. - i is a randomly generated integer.
- AngleStep (
M_AUG_ROTATION_OP_ANGLE_STEP) represents the step angle. The step angle is the distance, or gap, relative to the reference angle. The random integer i determines the number of rotation steps from the reference angle to reach the angle range from which to randomly select a rotation angle. - AngleDelta (
M_AUG_ROTATION_OP_ANGLE_DELTA) represents the delta angle. The delta angle is the angular range, relative to (AngleRef + i * AngleStep). - AngleMax (
M_AUG_ROTATION_OP_ANGLE_MAX) represents the maximum rotation angle.
If the generated random rotation angle is outside the range of AngleMin and AngleMax, another combination of i and RandomAngle will be generated. By default, AngleMin (M_AUG_ROTATION_OP_ANGLE_MIN) and AngleMax (M_AUG_ROTATION_OP_ANGLE_MAX) are set to 0 degrees and 360 degrees, respectively.
The following example illustrates how the random rotation angle is established. Note that since i is randomly selected, an angle can be selected from any of the angular ranges around each possible step (as long as the angle is between the specified minimum and maximum angle).
[Image: MimAugment_rotation_angle.png]