Kenneth: /* Docker Hub */

2023-12-15T19:33:21Z

Docker Hub

Kenneth: /* Exporting */

2023-07-31T20:55:21Z

Exporting

New page

== Docker ==

https://www.docker.com/

== Docker Hub ==

Docker Hub
https://hub.docker.com/

== What Is Docker ==

Package your application into a standardized unit for software development

:"Docker containers wrap a piece of software in a complete filesystem that contains everything needed to run: code, runtime, system tools, system libraries – anything that can be installed on a server. This guarantees that the software will always run the same, regardless of its environment."

https://www.docker.com/what-docker

--

"Docker is an open-source project that automates the deployment of applications inside software containers."

Docker provides an additional layer of abstraction and automation of operating-system-level virtualization on Linux. Docker uses the resource isolation features of the Linux kernel such as cgroups and kernel namespaces, and a union-capable file system such as OverlayFS and others to allow independent "containers" to run within a single Linux instance, avoiding the overhead of starting and maintaining virtual machines.

The Linux kernel's support for namespaces mostly isolates an application's view of the operating environment, including process trees, network, user IDs and mounted file systems, while the kernel's cgroups provide resource limiting, including the CPU, memory, block I/O and network. Since version 0.9, Docker includes the libcontainer library as its own way to directly use virtualization facilities provided by the Linux kernel, in addition to using abstracted virtualization interfaces via libvirt, LXC (Linux Containers) and systemd-nspawn.

As actions are done to a Docker base image, union file system layers are created and documented, such that each layer fully describes how to recreate an action. This strategy enables Docker's lightweight images, as only layer updates need to be propagated (compared to full VMs, for example).

https://en.wikipedia.org/wiki/Docker_(software)

== Removing all containers and images ==

# Stop the container(s) using the following command:
docker-compose down
# Delete all containers using the following command:
docker rm -f $(docker ps -a -q)
# Delete all volumes using the following command:
docker volume rm $(docker volume ls -q)
# Prune system images, included stopped:
docker system prune -a

Ref [https://docs.tibco.com/pub/mash-local/4.1.1/doc/html/docker/GUID-BD850566-5B79-4915-987E-430FC38DAAE4.html] [https://www.digitalocean.com/community/tutorials/how-to-remove-docker-images-containers-and-volumes]

== A Docker Tutorial for Beginners ==

A Docker Tutorial for Beginners<br>
https://docker-curriculum.com/

Once you are done installing Docker, test your Docker installation by running the following:
# docker run hello-world

hello-world container: https://hub.docker.com/_/hello-world

Example:
<pre>
root@server:~# docker run hello-world
Unable to find image 'hello-world:latest' locally
latest: Pulling from library/hello-world
b8dfde127a29: Pull complete
Digest: sha256:9f6ad537c5132bcce57f7a0a20e317228d382c3cd61edae14650eec68b2b345c
Status: Downloaded newer image for hello-world:latest

Hello from Docker!
This message shows that your installation appears to be working correctly.

To generate this message, Docker took the following steps:
1. The Docker client contacted the Docker daemon.
2. The Docker daemon pulled the "hello-world" image from the Docker Hub.
(amd64)
3. The Docker daemon created a new container from that image which runs the
executable that produces the output you are currently reading.
4. The Docker daemon streamed that output to the Docker client, which sent it
to your terminal.

To try something more ambitious, you can run an Ubuntu container with:
$ docker run -it ubuntu bash

Share images, automate workflows, and more with a free Docker ID:
https://hub.docker.com/

For more examples and ideas, visit:
https://docs.docker.com/get-started/
</pre>

Example of Ubuntu container:
<pre>
root@mserver:~# docker run -it ubuntu bash
Unable to find image 'ubuntu:latest' locally
latest: Pulling from library/ubuntu
c549ccf8d472: Pull complete
Digest: sha256:aba80b77e27148d99c034a987e7da3a287ed455390352663418c0f2ed40417fe
Status: Downloaded newer image for ubuntu:latest
root@4d827b59d198:/#
</pre>

Try Busybox container:
<pre>
## Fetch container
docker pull busybox

## List containers
docker images

## Run container (did nothing then exited)
docker run busybox

## Run container with instructions to do something
docker run busybox echo "hello from busybox"
hello from busybox

## List running docker containers:
docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES

## List all dockers containers that we have run:
docker ps -a

## Run in interactive mode:
docker run -it busybox sh

## Run and auto delete on exit
docker run --rm busybox echo "hello"

## Remove docker containers (clean up after yourself
docker rm [CONTAINER_ID CONTAINER_ID ...]

## Remove all exited docker containers:
docker rm $(docker ps -a -q -f status=exited)
# or in later editions
docker container prune

## Remove docker images:
docker rmi [IMAGE_ID IMAGE_ID ...]

## Run a detached instance, and name it
docker run -d --name [NAME] [IMAGE]
# example: docker run -d --rm --name u3 -it ubuntu bash

## Attach to running container: [https://docs.docker.com/engine/reference/commandline/attach/]
docker attach [NAME]

## Detach from a running container: [https://docs.docker.com/engine/reference/commandline/attach/]
CTRL-p CTRL-q

## Stop detached instance
docker stop [NAME]

## Pull a specific version:
docker pull ubuntu:latest
docker pull ubuntu:18.04
docker pull ubuntu:16.04

## Run specific version
docker run ubuntu:16.04
</pre>

<pre>
root@server:~# docker pull busybox
Using default tag: latest
latest: Pulling from library/busybox
b71f96345d44: Pull complete
Digest: sha256:930490f97e5b921535c153e0e7110d251134cc4b72bbb8133c6a5065cc68580d
Status: Downloaded newer image for busybox:latest
docker.io/library/busybox:latest

root@server:~# docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
ubuntu latest 9873176a8ff5 13 days ago 72.7MB
busybox latest 69593048aa3a 3 weeks ago 1.24MB
hello-world latest d1165f221234 3 months ago 13.3kB

root@server:~# docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
495cf502dde2 busybox "echo 'hello from bu…" 2 minutes ago Exited (0) 2 minutes ago
</pre>

=== Create Image ===

All user images are based on a base image.

Login to Docker Hub:
docker login

Sample Dockerfile:
<pre>
FROM python:3

# set a directory for the app
WORKDIR /usr/src/app

# copy all the files to the container
COPY . .

# install dependencies
RUN pip install --no-cache-dir -r requirements.txt

# define the port number the container should expose
EXPOSE 5000

# run the command
CMD ["python", "./app.py"]
</pre>

Simple Dockerfile:
<pre>
FROM ubuntu

RUN touch test.txt
</pre>

Build image:
docker build -t [username]/[project] .

Test imagE:
docker run [username]/[project]<:version>

Push image to Docker Hub:
docker push [username]/[project]

See parent hash of image:
docker history --format '{{ .ID }}' --no-trunc [USERNAME]/[PROJECT]

== Pull Image ==

docker pull

Hash specific: [https://docs.docker.com/engine/reference/commandline/pull/]
docker pull ubuntu@sha256:26c68657ccce2cb0a31b330cb0be2b5e108d467f641c62e13ab40cbec258c68d

Hash specific in Dockerfile: [https://docs.docker.com/engine/reference/commandline/pull/]
FROM ubuntu@sha256:26c68657ccce2cb0a31b330cb0be2b5e108d467f641c62e13ab40cbec258c68d

== Attach to Running Docker ==

docker attach [NAME]

If it isn't running bash, you can do something like this:
docker exec -it [NAME] /bin/bash

== Running ARM on x86 ==

'''Problem:'''
uname -m # Display the host architecture
#x86_64

docker run --rm -t arm64v8/ubuntu uname -m # Run an executable made for aarch64 on x86_64
#standard_init_linux.go:211: exec user process caused "exec format error"

'''Solution:'''

As expected the instructions are not recognized since the packages are not installed yet. Installing the following packages should allow you to enable support for aarch64 containers on your x86 workstation:

sudo apt-get install qemu binfmt-support qemu-user-static # Install the qemu packages
docker run --rm --privileged multiarch/qemu-user-static --reset -p yes # This step will execute the registering scripts

docker run --rm -t arm64v8/ubuntu uname -m # Testing the emulation environment
#aarch64

ref: [https://www.stereolabs.com/docs/docker/building-arm-container-on-x86/]

== Static MAC ==

docker run ... --mac-address 12:34:56:78:9a:bc

ref: [https://docs.docker.com/engine/reference/run/#:~:text=By%20default%2C%20the%20MAC%20address,specified%20MAC%20addresses%20are%20unique.]

== Exporting ==

=== Save and Load Image ===

docker save -o [image.tar] [IMAGE]
# or
docker save [IMAGE] > [image.tar]

docker load -i [image.tar]
# or
cat [image.tar] | docker load

=== Export and Import Container ===

docker export [CONTAINER_ID_OR_NAME] > [container.tar]

docker import [container.tar] [IMAGE_NAME:VER]
# or
cat [container.tar] | docker import - [IMAGE_NAME:VER]

== Extract Build Steps ==

docker history [IMAGE_ID] --format '{{.CreatedBy}}' --no-trunc | tac

docker history [IMAGE_ID] --no-trunc | tac

== Docker Compose ==

apt install docker-compose docker-compose-plugin
# docker-compose provides "docker-compose" command
# docker-compose-plugin provides "docker compose" command

docker compose up -d

Sample docker-compose.yml:
<pre>
version: "3.0" # compose API version
services:
license-service:
container_name: license
# Docker image name
image: flexlm:v1
build:
context: .
user: root
restart: always
# mac address tied to license file
mac_address: xx:xx:xx:xx:xx:xx
ports:
# 27000 for lmgrd
- 27000:27000
# 27001 for armlmd
- 27001:27001
</pre>

== Proxy ==

Client:
~/.docker/config.json

<pre>
{
"proxies": {
"default": {
"httpProxy": "http://proxy.example.com:3128",
"httpsProxy": "https://proxy.example.com:3129",
"noProxy": "*.test.example.com,.example.org,127.0.0.0/8"
}
}
}
</pre>

ref: Configure Docker to use a proxy server | Docker Documentation - https://docs.docker.com/network/proxy/

Configure Docker Service

systemctl edit docker.service

<pre>
[Service]
Environment="HTTP_PROXY=http://proxy.example.com:3128",
Environment="HTTPS_PROXY=httpS://proxy.example.com:3129",
Environment="NO_PROXY=localhost,127.0.0.1",
</pre>
systemctl restart docker.service

== Registry ==

=== Registry API ===

HTTP API V2 | Docker Documentation
https://docs.docker.com/registry/spec/api/

List available images:
GET /v2/_catalog
GET https://registry.example.com/v2/_catalog

Pretty print output:
curl -s https://registry.example.com/v2/_catalog | jq
curl -s https://registry.example.com/v2/_catalog | python3 -m json.tool

Pagenation
GET /v2/_catalog?n=<count>
# using a name of repo in a previous result:
GET /v2/_catalog?n=<count>&last=<name_repo_start_with>

List tags: (pagination also works with tags)
GET /v2/<image_name>/tags/list

Delete image:
# reference must be a digest or the delete will fail.
DELETE /v2/<name>/manifests/<reference>

API:
Method Path Entity Description
GET /v2/ Base Check that the endpoint implements Docker Registry API V2.
GET /v2/<name>/tags/list Tags Fetch the tags under the repository identified by name.
GET /v2/<name>/manifests/<reference> Manifest Fetch the manifest identified by name and reference where reference can be a tag or digest. A HEAD request can also be issued to this endpoint to obtain resource information without receiving all data.
PUT /v2/<name>/manifests/<reference> Manifest Put the manifest identified by name and reference where reference can be a tag or digest.
DELETE /v2/<name>/manifests/<reference> Manifest Delete the manifest or tag identified by name and reference where reference can be a tag or digest. Note that a manifest can only be deleted by digest.
GET /v2/<name>/blobs/<digest> Blob Retrieve the blob from the registry identified by digest. A HEAD request can also be issued to this endpoint to obtain resource information without receiving all data.
DELETE /v2/<name>/blobs/<digest> Blob Delete the blob identified by name and digest
POST /v2/<name>/blobs/uploads/ Initiate Blob Upload Initiate a resumable blob upload. If successful, an upload location will be provided to complete the upload. Optionally, if the digest parameter is present, the request body will be used to complete the upload in a single request.
GET /v2/<name>/blobs/uploads/<uuid> Blob Upload Retrieve status of upload identified by uuid. The primary purpose of this endpoint is to resolve the current status of a resumable upload.
PATCH /v2/<name>/blobs/uploads/<uuid> Blob Upload Upload a chunk of data for the specified upload.
PUT /v2/<name>/blobs/uploads/<uuid> Blob Upload Complete the upload specified by uuid, optionally appending the body as the final chunk.
DELETE /v2/<name>/blobs/uploads/<uuid> Blob Upload Cancel outstanding upload processes, releasing associated resources. If this is not called, the unfinished uploads will eventually timeout.
GET /v2/_catalog Catalog Retrieve a sorted, json list of repositories available in the registry.

== Reducing Image Size ==

1. Bundle Layers

RUN apt-get update -y \
&& apt install -y package_one

2. Avoid extra packages

apt install -y --no-install-recommends

3. Clean up after install

apt ... \
&& rm -rf /var/lib/apt/lists/*

4. Pick smaller image

alpine vs ubuntu

5. Use .dockerignore

ref: [https://towardsdatascience.com/5-easy-to-implement-tricks-to-trim-down-your-docker-image-size-263978a6ed29]

== Issues ==

=== apt getting stuck at configuring tzdata ===

<pre>
Configuring tzdata
------------------

Please select the geographic area in which you live. Subsequent configuration
questions will narrow this down by presenting a list of cities, representing
the time zones in which they are located.

1. Africa 4. Australia 7. Atlantic 10. Pacific 13. Etc
2. America 5. Arctic 8. Europe 11. SystemV
3. Antarctica 6. Asia 9. Indian 12. US
Geographic area:
</pre>

Solution:
<pre>
FROM ubuntu:20.04

ENV TZ=Asia/Dubai
RUN ln -snf /usr/share/zoneinfo/$TZ /etc/localtime && echo $TZ > /etc/timezone

RUN apt update
RUN apt install -y tzdata
</pre>

ref: Fix: tzdata hangs during Docker image build - DEV Community - https://dev.to/grigorkh/fix-tzdata-hangs-during-docker-image-build-4o9m

== keywords ==

@@ Line 7: / Line 7: @@
   Docker Hub
   https://hub.docker.com/
 == What Is Docker ==

Docker - Revision history

Kenneth: /* Docker Hub */

Kenneth: /* Exporting */