Friday, June 26, 2020

Common Docker, Dockerfile and Bash commands

Docker commands (in bash)

docker images
list all images available on local

docker rmi imageNameOrID
docker rmi imagesNameOrID -f 
remove a docker image from local
using -f to force remove image even if it is attached to some containers 

docker build -t imagename .
docker build-t imagename:version .
build a docker image and give it a specified name using the dockerfile in the current directory.
-t to tag the image to a particular name and version

docker ps -a -q
docker container ls
list all running containers, -a (all) to show all container, by default, only running container shows. -q only show container ID.
 

docker ps -a
docker container ls -a
list all running and stopped containers

docker kill containername
kill a running container by name

docker rm containername -f
remove a container (using -f option to force to remove running container)

docker run --name ContainerName -d -p 8000:80 Imagename
run a nginx image of imagename in detached mode (-d), with host port (-p)8000 maps to nginx container's default port 80. By default, the nginx listens on port 80 after starting the container.  Assuming the index.html is in the host's current folder, then it can be loaded from browser with the below link

docker run --rm -it imagename bash
--rm :automatically remove container when it exits
-it bash: interactively start a bash command console, start the container and also open a bash command console, 

docker start containername
docker stop containername
docker restart containername
start, stop, restart a container by name

docker attach containername
attach docker container to std to see the container's console output, using ctrl+c to exit.

docker exec -it containername /bin/bash
docker exec containername bashcommand
attach to container's bash console to run bash command inside container, using exit command to exit.
Once enter the bash command console, you can install other required software, for example, installing vim as below:
apt-get update
apt-get install vim
using below exec command to show all environment variables
docker exec containername env

docker cp container:SrcPath hostDestPath
docker cp hostSrcPath container:destPath
copy files between container and host


docker stop $(docker ps -aq)
stop all containers

docker rm $(docker ps -aq)
delete all containers

docker rmi $(docker images -q)
delete all docker images

Steps to push local docker image to docker hub
1. First login to docker.com using your docker id and password
docker login

2. Tag the local image with the namespace of your docker userid
docker image tag myimage:mytag myDockerUserID/myimage:mytag

3. push the tagged image to docker hub
docker image push myDockerUserID/myimage:mytag

DockerFile commands:

FROM nginx
FROM node:
download the base image from dockerhub or local

COPY . /usr/share/nginx/html
copy the static files from the current directory to the specified folder in docker container. The above sample copies files to container's /usr/share/nginx/html folder. 

ADD https://www.python.org/ftp/python/3.5.1/python-3.5.1.exe /temp/python-3.5.1.exe
copy files from source folder or external url to the destination folder in the container

WORKDIR directoryName
change the current directory in container

RUN shellCommand
run a shell command on the top of current generated image and commit to the updated image, for example,
RUN echo 'run docker file

ARG varnameWithoutDefaultValue
ARG varname=varDefaultValue
Set a variable that can be set by docker build, those variables are only used during docker build, not available when running the container.
Those variable can be set or replaced when running docker build with --build-arg flag
Example - the below code logs the variable DIST_DIR
FROM nginx:1.18
ARG DIST_DIR=dist
RUN echo 'DIST_DIR is set to' ${DIST_DIR}

ENV envname=envValue
ENV envName the environment value
Set environment variable for docker run, those variables are used during docker build and docker run. The ENV value can be replaced using docker run --env flag.
ENV USER_ID=12345 C
RUN echo 'User ID is set to' ${USER_ID}

ENTRYPOINT ["ExeBashName"]
CMD ["p1", "p2"]
Specify the executable name by EntryPoint, as well as default parameters by CMD. Unlike, Run, CMD does not execute anything at build time, but specifies the intended command when starting the image.



Nginx docker image notes
The default docker config file for nginx is in etc/nginx/config.d/default.conf
server {
    listen       80;
    listen  [::]:80;
    server_name  localhost;

    #charset koi8-r;
    #access_log  /var/log/nginx/host.access.log  main;

    location / {
        root   /usr/share/nginx/html;
        index  index.html index.htm;
    }

    #error_page  404              /404.html;

    # redirect server error pages to the static page /50x.html
    #
    error_page   500 502 503 504  /50x.html;
    location = /50x.html {
        root   /usr/share/nginx/html;
    }

    # proxy the PHP scripts to Apache listening on 127.0.0.1:80
    #
    #location ~ \.php$ {
    #    proxy_pass   http://127.0.0.1;
    #}

 
# pass the PHP scripts to FastCGI server listening on 127.0.0.1:9000 # #location ~ \.php$ { # root html; # fastcgi_pass 127.0.0.1:9000; # fastcgi_index index.php; # fastcgi_param SCRIPT_FILENAME /scripts$fastcgi_script_name; # include fastcgi_params; #} # deny access to .htaccess files, if Apache's document root # concurs with nginx's one # #location ~ /\.ht { # deny all; #} }

Command and Argument for docker

CMD
CMD ["nginx"] // example in nginx docker file
CMD ["bash"] //example in ubuntu docker file
specify the command that will run when docker container starts, CMD configuration can be overridden by docker run parameter
docker run ubuntu sleep 5
sleep 5 will replace default "bash" command specified in docker file. You can also create a new docker file to replace the CMD

CMD["sleep", "5"] //the first element in the array is command to execute. This command can be overridden by
docker run newubuntuimage sleep 10

If you only want to specify the command to execute, and leave argument to be overridden, then use ENTRYPOINT in docker file
ENTRYPOINT ["sleep"]
in that case, any docker run command line argument will be appended after the entrypoint command
docker run ubuntu-sleeper 100
will run sleep 100 in docker image

so, CMD will be replaced by command line argument, while ENTRYPOINT will be appended with command line argument.

When both CMD and ENTRYPOINT are used, then entrypoint command is the execution command, and the default argument is what specified in
CMD, CMD value can be overridden by the docker run's argument if provided.
ENTRYPOINT ["sleep"]
CMD ["5"]







Sunday, June 21, 2020

Understanding Azure system assigned and user assigned identity

For most Azure resource, like, Web App, Function App, Azure VM, etc,  there is an identity property under settings section, this identity property allows developer to set a system assigned identity or user assigned identity to an azure resource (i.e secret consumer). The identity can be used by other azure resources (secret holder) to assign permissions to the secret consumers represented by the identity. Usually the secret consume is an .net core web app or a java spring web app, and secret holder is a azure keyvault, which holds the password or connection strings to database or azure storage account.

When system assigned identity is enabled, azure creates an principal ID to represent this azure resource (secret consume), each azure resource can only have one system assigned identity. Other azure resources (secret holder) can set access policies to the principal id of system assigned identity, similar to assigning permission to a role based user account, so that it enables the system assigned identity to send requests to the secure holder resources. 

For example, in Azure keyVault, under Settings' Access Policies blade, if permission is assigned to the system identity for a VM or web app, the assigned permission will show under the current access policy list. 

A typical use case involves a SPA client app, a java spring or .net core web app, datavault, and database or Azure storage account. The SPA app runs on client side and does not hold any sensitive data. The SPA app sends request to java spring or .net web app, which as a system assigned identity to allow it to access the datavault through an access policy defined in datavault, so that the web app itself does not need to save any sensitive data in its code. Also no direct trust relationships need to be configured between web app and database or storage account. The only trust relationship that needs to be configured is between the web app and datavault.

As each azure service can only has one system assigned identity, it may not be enough. In that case, multiple user assigned identities can be created. Those user assigned identities can be associated with any azure resources, and then they can be used in the same way as system assigned identity for assigning permission from other azure resources.

The system assigned identity can be enabled from azure portal or az portal shell. The below command creates a system identity for a azure web app 

az webapp identity assign --name yourWebappName --resource-group yourResourceGroupName

The output includes the principal identity as below

{

  "principalId": "1c36874b-3a68-47b5-88ed-4b1ef9ee45b7",

  "tenantId": "7fe7fa7d-cac4-43e5-8f35-eec8db5a662f",

  "type": "SystemAssigned",

  "userAssignedIdentities": null

}

In order to get permission to azure resource (like a key vault), you can set the permission from the azure resource (secret holder) as below

az keyvault set-policy --name haiquankeyvault --object-id 1c36874b-3a68-47b5-88ed-4b1ef9ee45b7 --secret-permissions get list

After the command, the access policy of the keyvault resource will include a new item for allowed permission assigned to the web app.


One particular use case for Azure managed identity is for Azure virtual machine, as once the permission is assigned to the VM's system identity, then any apps or services running on this Virtual machine can transparently get this VM identity's permission to access the assigned resources without providing any credentials or access key information.

The below command creates a system assigned identity for an Azure virtual machine.

az vm identity assign --name myVM --resource-group myResourceGroup

The output of the command is the identity of the vm as below:

{

  "systemAssignedIdentity": "d5b1bb44-a5b5-4eeb-9c68-920a385d310c",

  "userAssignedIdentities": {}

}

The identity created by the above commands can represent the VM to assign permission to this VM identity. The below code assigning Azure keyvault permission to the system assigned identity created before.

az keyvault set-policy --name haiquankeyvault --object-id d5b1bb44-a5b5-4eeb-9c68-920a385d310c --secret-permissions backup, delete, get, list, purge, recover, restore, set

The below code shows how a .net core apps running in the VM can use SecretClient and DefaultAzureCredential instance s to get the access to the keyvault without providing any credentials.

        static void Main(string[] args)
        {
            string secretName = "mySecret";
            string keyVaultName = "haiquankeyvault";

            var kvUri = "https://haiquankeyvault.vault.azure.net";

            var client = new SecretClient(new Uri(kvUri), new DefaultAzureCredential());

            Console.Write("Input the value of your secret > ");
            string secretValue = Console.ReadLine();

            client.SetSecret(secretName, secretValue);

            Console.WriteLine(" done.");
}

Wednesday, June 17, 2020

Oauth2 authentication flow difference between browser web app, SPA web app, and mobile/desktop application

The basic oauth2 flow is similar between browser app, Single page app, and mobile/desktop, where user need to input username and password to get authorization code from id provider, and then using the authorization code to exchange to access token and refresh token. Finally the access token can be used to retrieve the data from resource holder. 

For all oAuth2 flow types, the username and password, and authorization code are not stored in anywhere, and they will be discarded immediately after their usage. So they do not cause any difference between different Oauth flow types.

However, client secret (obtained during oauth app registration), access token and refresh tokens (obtained during oauth authentication) are sensitive data, and need to be securely stored in somewhere. The issue is some client types (like browser) cannot securely store the confidential data as other types (like web server or native app), this difference leads to the various Oauth2 authentication types, which are specific to Web App (Client requests proxy through web server to Web API), Single Page App (client directly request to WebAPI without web server), and Mobile/Desktop native app.  

Browser Web App (using web server as proxy to WebAPI)
As browser is considered not secure for storing confidential data, so it cannot be used to store the client secret, access token and refresh token. As  a result, once the user input the username and password, and get the authorization code, the browser will redirect the url to the web server app, and the web server will use the authorization code as well as the securely stored client secret to request the access token and refresh token, once the web server gets the access token, it will request the resource using the access token, and then send the resource data back to browser client. The client secret, access token and refresh tokens are securely stored on web server, and are never exposed to browser. The diagram is shown as below

Single Page App
There is a special case for SPA (Single Page App) browser app, like angular or react web app, as SPA app directly access the WebAPI server without proxy thorough web server as middle tier, so it cannot securely store the client secret, access token and refresh token. 

First, as SPA cannot securely store the client secret on client, the best way to do is not using secret at all, which is why oAuth Implicit authentication flow should be used for SPA app.

In this case, many applications choose to store the token in memory to reduce the risk of leaking token. However, a better approach is using implicit grand in this situation, so browser only gets the access token without refresh token, this approach is described at 
Diagram showing the implicit sign-in flow


Mobile/Native App
For mobile/desktop applications, the compiled installation are not regarded as secure, so it cannot be used to store client secret, that is why client secret should not be used for mobile/native app. However, mobile app are considering secure for storing runtime confidential data, because they can protect the sensitive data by password, finger print, or other device security methods, so the native app can store the access token and refresh token on client side, and there is no need to store them on server side. The diagram is shown as below: