Containerized Selenium Browser Automation Architecture With Azure Functions .NET 3.1 | Pt. 2: Deployment of a Stateless Container Instance

Table Of Contents

Overview

If you just got here, check out the first part of this tutorial here.

So, we have our automation ready and it purrs like a kitten when on our machine. All we need now is deploying it to Azure!

We’ll need:

• Azure Functions App
• Azure Container Instance
• Docker Hub image (or Azure Container Registry)
• Azure Storage Account
• Azure File Share

Worry not! It might seem overwhelming, but it’s really easy in reality!

First, a few words on persisting a browser state. As we read in MS docs:

By default, Azure Container Instances are stateless. If the container is restarted, crashes, or stops, all of its state is lost. To persist state beyond the lifetime of the container, you must mount a volume from an external store.

Mount an Azure file share in Azure Container Instances

What this means for our browser automation is that any websites we log in to, will now log us out, all cookies and cache get removed. In short, the state is lost.

The main advantage of stateful automation is the ability to keep our accounts logged in, 2-factor authentication is no longer an issue, and our web activity all together looks less suspicious to the websites we visit and know this – websites don’t like suspicious activity.

Good and bad news on this: good – it’s possible to keep the state, bad – it’s a little bit convoluted. To keep things simple, we’ll lay a foundation for keeping the state (deploy the container with Azure File Share volume mounted), but we won’t quite get there in this article (and get this sorted in the next one).

Pushing Image to Docker Hub

On your machine, open Command Prompt and type:

docker images

Copy the Image ID of your container of interest and type the following:

docker run YOUR_IMAGE_ID

You now have a running container. You can change the state of it the desired one, but we don’t need to do anything at this stage (our state will be persisted by a file system and not the actual state of the container). We can save it as a new image. Keep the convention DOCKER_ACCOUNT_ID/CONTAINER_NAME (ex. fullduckdev/selenium). You can do this by running:

docker container ls

Note the Container ID of the container you just ran/amended and then:

docker container commit CONTAINER_ID CONTAINER_NAME

if you now type `docker ps`, you should see that new container on the image list. Perfect! Now, it’s time to push it to the Docker repository (you could also use Azure Container Registry). You can do that by running:

docker image push DOCKER_ACCOUNT_ID/CONTAINER_NAME

It now should be available in Docker Hub when you log in (I named mine pawelflajszer/seleniumsample):

Deploying Azure File Share

Create your File Share using the guide here or follow the below quick start.

First, create a storage account in Azure. Navigate to Azure Cloud Shell (or login to Azure CLI on your machine), then run the bellow snippet. Remember to specify your subscription ID (if you have multiple subscriptions like me)

$rg="browser-automation"
$subscription=YOUR_ID
$location = "westeurope"
$fileshare_name = "browser-automation-fs"
$storage_account = "browserautomation"
az storage account create --resource-group $rg --name $storage_account --location $location --sku Standard_LRS --subscription $subscription

Next, create a file share on that Storage Account by running the below:

az storage share create --name $fileshare_name --account-name $storage_account --subscription $subscription

It should now appear under your storage account > file share node like so:

Deploying Container Instance with Azure File Share

First, we need storage account credentials, we can get them by running the below in the shell:

$STORAGE_KEY = $(az storage account keys list --resource-group $rg --account-name $storage_account --subscription $subscription  --query "[0].value" --output tsv)

Next, we need to mount the File Share on deployment (can’t do it post-deployment). Please note that below, we’re using the DNS flag to specify the fixed address for our container (FQDN) and we deliberately want to open both 80 (Web client) and 4444 (Selenium) ports:

$container_name = "selenium-instance"
$image = "pawelflajszer/seleniumsample"
$dns = "selenium-instance"
$fs_mount_path = "/home/seluser/azurefileshare"

az container create --resource-group $rg --name $container_name --image $image --dns-name-label $dns --ports 80 4444 --subscription $subscription --azure-file-volume-account-name $storage_account --azure-file-volume-account-key $STORAGE_KEY --azure-file-volume-share-name $fileshare_name --azure-file-volume-mount-path $fs_mount_path

That’s it! Our Container Instance should now be available to use in Azure Portal. You can inspect the filesystem in the App Service > Containers > Connect (you can run bash scripts there).

Properties tab should tell you the open ports and if and which Azure File Share is mounted in, with a mount path specified. My path was /home/seluser/azurefileshare and I can prove this location exists when navigating there in the console (in the Connect tab). At this point, you can upload files (i.e. via Azure Storage Explorer) and those should appear in that folder inside the container.

Deployed container GUI is available under your FQDN:4444 (FQDN is the address of your container, available in the portal).

Deploying Azure Functions App

Deploying an empty Functions app can’t be easier. Let’s introduce some variables (you can change the name and leave the rest as below). We’ll also be re-using some earlier declared variables, so if you have a new Cloud Shell Instance open, make sure to declare $subscription, $rg (resource group name) and $storage_account.

$func_name = "browserautomationfunc"
$runtime = "dotnet"
$func_ver = "3"

az functionapp create --resource-group $rg --consumption-plan-location $location --runtime $runtime --functions-version $func_ver --name $func_name --storage-account $storage_account --subscription $subscription

We’ll skip the source control and CD pipeline for the sake of simplicity and publish the app straight from Visual Studio.

1. Right Click on your Azure Functions project
2. Select Publish…
3. Select Azure
4. Select Azure Function App (Windows)
5. Select your subscription and search folder structure for youir Func app (mine is browserautomationfunc)
6. Make sure Run from package file box is selected
7. Press Next
8. Select Publish
9. Press Finish
10. Press Publish

You can prove your Azure Functions app is deployed by navigating to Azure Portal and selecting the Functions pane, you should see all your functions there.

To run any of the functions using the container we’ve deployed, you’ll have to add configuration variables from your local.settings.json to the app service (Configuration pane).

Make sure to replace this one:

"containerUrl": "http://localhost:4444/wd/hub"

with the correct, deployed container. That’s going to be your FQDN. It’s also available in the portal > Your Container instance overview. Mine is

"containerUrl": "http://selenium-instance.westeurope.azurecontainer.io:4444/wd/hub"

Conclusion

We now have a container with Selenium image and an active Azure Function both running in the cloud. Every time the container runs, the state of the browser is reset and a new Chrome profile is being created. This can pose an issue when logging in to certain services, as they’re no fan of robots/selenium users. We’ll tackle that problem next.