CameraX on Android Fragment in Kotlin with ImageAnalyzer

CameraX

CameraX is a relatively new Camera API in Android. Unlike the predecessors, CameraX is easy to use and requires less tweaking in order to adopt a Camera View in your application. Since, in the official sample code, Google made it a little buggy (for good intentions though) and they only implement it on Activity. So in this article, I’ll go over how to make implement CameraX using Fragment and mention some of the points we should be caution with.

1. Importing CameraX project in Gradle.

As of the day i write this article, the latest version of CameraX is “1.0.0-beta06”

def camerax_version = "1.0.0-beta06"
// CameraX core library using camera2 implementation
    implementation "androidx.camera:camera-camera2:$camerax_version"
// CameraX Lifecycle Library
    implementation "androidx.camera:camera-lifecycle:$camerax_version"
// CameraX View class
    implementation "androidx.camera:camera-view:1.0.0-alpha13"

Also, remember to add the below block right below buildTypes in your module/app Gradle, in order for CameraX to use the Java 8 feature

compileOptions {
    sourceCompatibility JavaVersion.VERSION_1_8
    targetCompatibility JavaVersion.VERSION_1_8
}

2. Add the PreviewView to your fragment layout

After creating the fragment and stuffs, add the camerax previewview to the layout like this:

<androidx.camera.view.PreviewView
    android:id="@+id/viewFinder"
    android:layout_width="match_parent"
    android:layout_height="match_parent" />

I made the width and height both match_parent to fit my use case here

3. Initialising the Camera and PreviewView

There are a few things to note here.

i. ImageCapture

If you intend to capture image, you need to initialise the ImageCapture this is method.

imageCapture = ImageCapture.Builder().build()

ii. ImageAnalyzer

If you intend to capture image, you need to initialise the ImageAnalyzer this is method.

imageAnalyzer = ImageAnalysis.Builder().build().apply {
        setAnalyzer(Executors.newSingleThreadExecutor(), CornerAnalyzer {
            val bitmap = viewFinder.bitmap
            val img = Mat()
            Utils.bitmapToMat(bitmap, img)
            bitmap?.recycle()

            val corner = processPicture(img)
            // Image analysis here
        })
    }

iii. Bind everything to life cycle

Once you need all these helper objects, you need to bind it one-by-one to the camera object.

camera = cameraProvider.bindToLifecycle(this, cameraSelector, imageAnalyzer, preview, imageCapture)

Full function code:

val cameraProviderFuture = ProcessCameraProvider.getInstance(safeContext)
cameraProviderFuture.addListener(Runnable {
    // Used to bind the lifecycle of cameras to the lifecycle owner
    val cameraProvider: ProcessCameraProvider = cameraProviderFuture.get()

    // Preview
    preview = Preview.Builder().build()

    imageCapture = ImageCapture.Builder().build()

    imageAnalyzer = ImageAnalysis.Builder().build().apply {
        setAnalyzer(Executors.newSingleThreadExecutor(), CornerAnalyzer {
            val bitmap = viewFinder.bitmap
            val img = Mat()
            Utils.bitmapToMat(bitmap, img)
            bitmap?.recycle()

            val corner = processPicture(img)
            // Image analysis here
        })
    }
    // Select back camera
    val cameraSelector = CameraSelector.Builder().requireLensFacing(CameraSelector.LENS_FACING_BACK).build()

    try {
        // Unbind use cases before rebinding
        cameraProvider.unbindAll()

        // Bind use cases to camera
        camera = cameraProvider.bindToLifecycle(this, cameraSelector, imageAnalyzer, preview, imageCapture)
        preview?.setSurfaceProvider(viewFinder.createSurfaceProvider())
    } catch (exc: Exception) {
        Log.e(TAG, "Use case binding failed", exc)
    }

}, ContextCompat.getMainExecutor(safeContext))

4. Setting a fixed Execution Thread

CameraX requires developer to set a thread for the execution. Set it in onViewCreated is a good place to choose.

cameraExecutor = Executors.newSingleThreadExecutor()

5. ImageAnalyzer

If you need to go image analysis, you need an image analyzer to provide you a bridge to get into the raw image data for every frame.

private class CornerAnalyzer(private val listener: CornersListener) : ImageAnalysis.Analyzer {

    private fun ByteBuffer.toByteArray(): ByteArray {
        rewind()    // Rewind the buffer to zero
        val data = ByteArray(remaining())
        get(data)   // Copy the buffer into a byte array
        return data // Return the byte array
    }

    @SuppressLint("UnsafeExperimentalUsageError")
    override fun analyze(imageProxy: ImageProxy) {
        if (!isOffline) {
            listener()
        }
        imageProxy.close()
    }

}

One thing to note here is that it is NOT recommended to do analysis here, because converting ImageProxy to bitmap is too much work. I just analyze the image at the end of the listener, which is so much easier.

6. Full Code in Kotlin

Now without further ado, let’s check out the complete code:

That’s it.

Happy coding.

Let me know if you find better solution or approach in the comment section below. ;)

Find me at Twitter @rick3817

Reference:

https://developer.android.com/training/camerax

Getting Started with CameraX

#camerax #android #androidcamera #imageanalysis