From 1fed33444518b413bf69491c2895d9c3e8b72ebb Mon Sep 17 00:00:00 2001 From: Teck Meng Date: Thu, 22 Aug 2024 08:43:02 +0800 Subject: [PATCH] Deployed 98303c6 with MkDocs version: 1.6.0 --- index.html | 2 +- resume/index.html | 12 ++++++++---- search/search_index.json | 2 +- sitemap.xml.gz | Bin 127 -> 127 bytes 4 files changed, 10 insertions(+), 6 deletions(-) diff --git a/index.html b/index.html index 171dece..7baff27 100644 --- a/index.html +++ b/index.html @@ -372,5 +372,5 @@

Objective

diff --git a/resume/index.html b/resume/index.html index 7362259..d9db328 100644 --- a/resume/index.html +++ b/resume/index.html @@ -359,7 +359,7 @@

Employment history : < --> -

+

https://www.uparcel.sg/

UParcel : 2023 - present

AI Engineer

-

+

https://aisingapore.org/

AI Singapore : 2022 - 2023

Associate AI Engineer

Career transition : 2021 - 2022 : 1 yrs

+

Team Fight Tactics Strategy Application

github.com/furyhawk/tftchamp

+

https://www.bhp.com/

NCS : 2020 - 2021 : 1 yrs

Fiori developer

-

BHP : 2019 - 2020 : 1 yr 2 mos

Developer

diff --git a/search/search_index.json b/search/search_index.json index dfcc1f9..e1cf565 100644 --- a/search/search_index.json +++ b/search/search_index.json @@ -1 +1 @@ -{"config":{"indexing":"full","lang":["en"],"min_search_length":3,"prebuild_index":false,"separator":"[\\s\\-]+"},"docs":[{"location":"","text":"Welcome to 124c41 type: short summary \u2502 \u2502 \u2502 \u2514\u2500\u2af8 Summary in present tense. Not capitalized. \u2502 \u2514\u2500\u2af8 Commit Type: build|cicd|docs|feat|fix|node|refactor|test AGI Humanity achieve AGI. Basic computing skills lost. NSFW content filtered for everyone. 1 Human decide to learn computing skills. NSFW content accessed. Home Server Chat : Chat with AI, RAG chatbot. Bot : AI assistant using GROQ, llama3 70B, RAG and web search. Stock Analysis Assistant : AI assistant using GROQ and llama3. ~~ Redlib : Reddit libre.~~ (killed by bots) Blog Ghost : Ghost blog. Beyond All Information : analyse your Beyond All Reason games. CheatSheets : Collection of cheatsheets. Cookbook : Collection of tech recipes. ~~ Forum : Host your own forum.~~ (deprecated) Neural Network Playground : Understand neural network visually. Note : Notepad Online. Use cookie storage only. linx : Image pastebin. pastebin : text/file Pastebin. speedtest : Speedtest. Home server : Build for ARM64 platform using Docker swarm mode. Team Fight Tactics ML : Analyse the current meta. Team Fight Tactics Strategy Application http://tftchamp.duckdns.org:3000/ Datasets publish @ https://www.kaggle.com/datasets/teckmengwong/team-fight-tactics-matches furyhawk/tftchamp: teamfight-tactics Data Analysis (github.com) About this dataset Team Fight Tactics highest ELO challengers games scrape by https://github.com/furyhawk/tftchamp. Using https://developer.riotgames.com/ API. 8 players FFA in one game. Target Label : placement 1 is best. Lower is better. Top 4 placement is a Win. Alternative prediction is to group Top 4 placement as Binary Win, bottom 4 as Binary Lost. Only team traits and augments/items chosen included in datasets. Stats like game_length , players_eliminated are excluded. This is to prevent the model from learning obvious predictor. sudo ./scripts/run_pipeline.sh -nrci Web Scraping With Python Objective This tutorial aims to show how to use the Python programming language to web scrape a website. Specifically, we will use the requests and Beautiful Soup libraries to scrape and parse data from companiesmarketcap.com and retrieve the \u201c Largest Companies by Market Cap \u201d. Finance details are scrape and parse from finance.yahoo.com . We will learn how to scale the web scraping process by first retrieving the first company/row of the table, then all companies on the website\u2019s first page, and finally, all 6024 companies from multiple pages. Once the scraping process is complete, we will preprocess the dataset and transform it into a more readable format before using matplotlib to visualise the most important information.","title":"Home"},{"location":"#welcome-to-124c41","text":"type: short summary \u2502 \u2502 \u2502 \u2514\u2500\u2af8 Summary in present tense. Not capitalized. \u2502 \u2514\u2500\u2af8 Commit Type: build|cicd|docs|feat|fix|node|refactor|test","title":"Welcome to 124c41"},{"location":"#agi","text":"Humanity achieve AGI. Basic computing skills lost. NSFW content filtered for everyone. 1 Human decide to learn computing skills. NSFW content accessed.","title":"AGI"},{"location":"#home-server","text":"Chat : Chat with AI, RAG chatbot. Bot : AI assistant using GROQ, llama3 70B, RAG and web search. Stock Analysis Assistant : AI assistant using GROQ and llama3. ~~ Redlib : Reddit libre.~~ (killed by bots) Blog Ghost : Ghost blog. Beyond All Information : analyse your Beyond All Reason games. CheatSheets : Collection of cheatsheets. Cookbook : Collection of tech recipes. ~~ Forum : Host your own forum.~~ (deprecated) Neural Network Playground : Understand neural network visually. Note : Notepad Online. Use cookie storage only. linx : Image pastebin. pastebin : text/file Pastebin. speedtest : Speedtest. Home server : Build for ARM64 platform using Docker swarm mode. Team Fight Tactics ML : Analyse the current meta.","title":"Home Server"},{"location":"#team-fight-tactics-strategy-application","text":"http://tftchamp.duckdns.org:3000/","title":"Team Fight Tactics Strategy Application"},{"location":"#datasets","text":"publish @ https://www.kaggle.com/datasets/teckmengwong/team-fight-tactics-matches furyhawk/tftchamp: teamfight-tactics Data Analysis (github.com)","title":"Datasets"},{"location":"#about-this-dataset","text":"Team Fight Tactics highest ELO challengers games scrape by https://github.com/furyhawk/tftchamp. Using https://developer.riotgames.com/ API. 8 players FFA in one game. Target Label : placement 1 is best. Lower is better. Top 4 placement is a Win. Alternative prediction is to group Top 4 placement as Binary Win, bottom 4 as Binary Lost. Only team traits and augments/items chosen included in datasets. Stats like game_length , players_eliminated are excluded. This is to prevent the model from learning obvious predictor. sudo ./scripts/run_pipeline.sh -nrci","title":"About this dataset"},{"location":"#web-scraping-with-python","text":"","title":"Web Scraping With Python"},{"location":"#objective","text":"This tutorial aims to show how to use the Python programming language to web scrape a website. Specifically, we will use the requests and Beautiful Soup libraries to scrape and parse data from companiesmarketcap.com and retrieve the \u201c Largest Companies by Market Cap \u201d. Finance details are scrape and parse from finance.yahoo.com . We will learn how to scale the web scraping process by first retrieving the first company/row of the table, then all companies on the website\u2019s first page, and finally, all 6024 companies from multiple pages. Once the scraping process is complete, we will preprocess the dataset and transform it into a more readable format before using matplotlib to visualise the most important information.","title":"Objective"},{"location":"about/","text":"Cervix defendite in atque Et Caesar ignes aspera Polymestoris dilectos obvius Lorem markdownum veli fratri tum illum coeptis, plagae? In armiferae rogum. Quas orsa Aeacide digitis huic solent moenia properata saetis, exitus dare gerunt viget ambage hac tamen terrae longe. Aura alvum, e requiescere, inrita ille foret cedere ego siquid deprensa culpetne ausa reparet, epota aequor ope . Ad sumptas medio , est cecidit more coniuge, viam. Est signaque parte, rogumque ensem, ubi bracchia Armeniae petis. Est nunc dicta terrae noluit. Nisi et caeco speciem expellam dolens praeceps Monstri collibus preces. Flavescunt tenuere Aegeus. click = alignment_point(rupSnapshot(vleOptic)) + uddiTooltipSnmp; if (leopardDriveE <= upTypeface(touchscreen - dvi, footer, cBootText.bar_heat.server(xhtmlWebcamAsp))) { compatibleMultithreadingTorrent.menu_pci(desktop_in, xmp_word, chipMashupDriver); browserUser = hypermedia_digitize_checksum(gigaflops_logic_winsock, cmos_pcb / clock_cron_drag, processor_cycle( keyboardOsJavascript, tweakCellListserv, control)); } type_kilohertz.standby_ram(microphone_null_osi, metaMedia); Adhibent cuius gentesque durasse dixit sterilique Meis me saxo; corpore pedibus. Undis Diomede! Se tellus ut sic illa et facti quamquam qualis fraudesque , conditus. Ulmo rurigenae proles prosiluit et plus movit. Titubantem inpono. Animalia pede, Minos nisi. Dixi stabat, curam curat vota sanguine Laestrygonis, ab materque finierat audes Thaumantias, terrae ulla, ego cetera. Serpentibus omnes exspectatum videre Aeneae thyrsos undae pietatis ulterius trementi agendum crudelis domitae. Fodiebant Caenis altae et Caeneus omnia ardor Consolor orbem tumentem, anguigenae sanguine tectis, lea glaebam guttae fuit valens caput, desubito te cursus aegre. Momordit misit solidumve Cereris cornu, illo dubiae: me est patris vias, mihi. Temptat spatiosi, cornu mater: cum Iris deus conchae tellusAndros tellus coniunx: sedes . Causa natorumque perque deponendique motu hoc facitote, lea quid quorum et multi venisse vox epulis nurusque? Duobus et corpora inerti, dea calamis equi quo iuncti thalamique starent suis est placido euntem ecce tertia. Nunc terrae flammas nec fuit minimamque, effugit ecce fulgentis in inter fertur: est sic colles ponderibus.","title":"About"},{"location":"about/#cervix-defendite-in-atque","text":"","title":"Cervix defendite in atque"},{"location":"about/#et-caesar-ignes-aspera-polymestoris-dilectos-obvius","text":"Lorem markdownum veli fratri tum illum coeptis, plagae? In armiferae rogum. Quas orsa Aeacide digitis huic solent moenia properata saetis, exitus dare gerunt viget ambage hac tamen terrae longe. Aura alvum, e requiescere, inrita ille foret cedere ego siquid deprensa culpetne ausa reparet, epota aequor ope . Ad sumptas medio , est cecidit more coniuge, viam. Est signaque parte, rogumque ensem, ubi bracchia Armeniae petis. Est nunc dicta terrae noluit.","title":"Et Caesar ignes aspera Polymestoris dilectos obvius"},{"location":"about/#nisi-et-caeco-speciem-expellam-dolens-praeceps","text":"Monstri collibus preces. Flavescunt tenuere Aegeus. click = alignment_point(rupSnapshot(vleOptic)) + uddiTooltipSnmp; if (leopardDriveE <= upTypeface(touchscreen - dvi, footer, cBootText.bar_heat.server(xhtmlWebcamAsp))) { compatibleMultithreadingTorrent.menu_pci(desktop_in, xmp_word, chipMashupDriver); browserUser = hypermedia_digitize_checksum(gigaflops_logic_winsock, cmos_pcb / clock_cron_drag, processor_cycle( keyboardOsJavascript, tweakCellListserv, control)); } type_kilohertz.standby_ram(microphone_null_osi, metaMedia);","title":"Nisi et caeco speciem expellam dolens praeceps"},{"location":"about/#adhibent-cuius-gentesque-durasse-dixit-sterilique","text":"Meis me saxo; corpore pedibus. Undis Diomede! Se tellus ut sic illa et facti quamquam qualis fraudesque , conditus. Ulmo rurigenae proles prosiluit et plus movit. Titubantem inpono. Animalia pede, Minos nisi. Dixi stabat, curam curat vota sanguine Laestrygonis, ab materque finierat audes Thaumantias, terrae ulla, ego cetera. Serpentibus omnes exspectatum videre Aeneae thyrsos undae pietatis ulterius trementi agendum crudelis domitae.","title":"Adhibent cuius gentesque durasse dixit sterilique"},{"location":"about/#fodiebant-caenis-altae-et-caeneus-omnia-ardor","text":"Consolor orbem tumentem, anguigenae sanguine tectis, lea glaebam guttae fuit valens caput, desubito te cursus aegre. Momordit misit solidumve Cereris cornu, illo dubiae: me est patris vias, mihi. Temptat spatiosi, cornu mater: cum Iris deus conchae tellusAndros tellus coniunx: sedes . Causa natorumque perque deponendique motu hoc facitote, lea quid quorum et multi venisse vox epulis nurusque? Duobus et corpora inerti, dea calamis equi quo iuncti thalamique starent suis est placido euntem ecce tertia. Nunc terrae flammas nec fuit minimamque, effugit ecce fulgentis in inter fertur: est sic colles ponderibus.","title":"Fodiebant Caenis altae et Caeneus omnia ardor"},{"location":"anomaly_detection/","text":"anomaly_detection pip install -e . pip install GitPython pip install onnx pip install openvino-dev Deep Learning for Anomaly Detection: A survey 1901.03407.pdf (arxiv.org) https://arxiv.org/pdf/1901.03407.pdf Type of Anomaly Point Anomalies represent an irregularity or deviation that happens randomly and may have no particular interpretation. Contextual Anomaly Detection conditional anomaly is a data instance that could be considered as anomalous in some specific context. Contextual anomaly is identified by considering both contextual and behavioural features. The contextual features, normally used are time and space. While the behavioral features may be a pattern of spending money, the occurrence of system log events or any feature used to describe the normal behavior. Collective or Group Anomaly Detection. Anomalous collections of individual data points are known as collective or group anomalies, wherein each of the individual points in isolation appears as normal data instances while observed in a group exhibit unusual characteristics. Output of DAD Techniques Anomaly Score Labels Diversity-Measurable Anomaly Detection | Papers With Code https://paperswithcode.com/paper/diversity-measurable-anomaly-detection Improvement of Autoencoders and Generative Adversarial Networks. Limitations . Focuses on anomaly with measurable geometrical diversity, the most common type in anomaly detection. However, as for anomaly with other kind of diversities, e.g. colors, the proposed diversity measure may not be positively correlated to anomaly severity. Attribute-based Representations for Accurate and Interpretable Video Anomaly Detection | Papers With Code https://paperswithcode.com/paper/attribute-based-representations-for-accurate Video anomaly detection Use object detection to extract features: Velocity Pose Deep features(pre-trained CLIP) Use kNN to detect anomaly on the extracted features. Could use similar approach to extract detective features. DSR -- A dual subspace re-projection network for surface anomaly detection | Papers With Code https://paperswithcode.com/paper/dsr-a-dual-subspace-re-projection-network-for Proposes an architecture based on quantized feature space representation with dual decoders, DSR, that avoids the image-level anomaly synthesis requirement. Without making any assumptions about the visual properties of anomalies, DSR generates the anomalies at the feature level by sampling the learned quantized feature space, which allows a controlled generation of near-in-distribution anomalies.","title":"anomaly_detection"},{"location":"anomaly_detection/#anomaly_detection","text":"pip install -e . pip install GitPython pip install onnx pip install openvino-dev","title":"anomaly_detection"},{"location":"anomaly_detection/#deep-learning-for-anomaly-detection-a-survey","text":"1901.03407.pdf (arxiv.org) https://arxiv.org/pdf/1901.03407.pdf","title":"Deep Learning for Anomaly Detection: A survey"},{"location":"anomaly_detection/#type-of-anomaly","text":"Point Anomalies represent an irregularity or deviation that happens randomly and may have no particular interpretation. Contextual Anomaly Detection conditional anomaly is a data instance that could be considered as anomalous in some specific context. Contextual anomaly is identified by considering both contextual and behavioural features. The contextual features, normally used are time and space. While the behavioral features may be a pattern of spending money, the occurrence of system log events or any feature used to describe the normal behavior. Collective or Group Anomaly Detection. Anomalous collections of individual data points are known as collective or group anomalies, wherein each of the individual points in isolation appears as normal data instances while observed in a group exhibit unusual characteristics. Output of DAD Techniques Anomaly Score Labels","title":"Type of Anomaly"},{"location":"anomaly_detection/#diversity-measurable-anomaly-detection-papers-with-code","text":"https://paperswithcode.com/paper/diversity-measurable-anomaly-detection Improvement of Autoencoders and Generative Adversarial Networks. Limitations . Focuses on anomaly with measurable geometrical diversity, the most common type in anomaly detection. However, as for anomaly with other kind of diversities, e.g. colors, the proposed diversity measure may not be positively correlated to anomaly severity.","title":"Diversity-Measurable Anomaly Detection | Papers With Code"},{"location":"anomaly_detection/#attribute-based-representations-for-accurate-and-interpretable-video-anomaly-detection-papers-with-code","text":"https://paperswithcode.com/paper/attribute-based-representations-for-accurate Video anomaly detection Use object detection to extract features: Velocity Pose Deep features(pre-trained CLIP) Use kNN to detect anomaly on the extracted features. Could use similar approach to extract detective features.","title":"Attribute-based Representations for Accurate and Interpretable Video Anomaly Detection | Papers With Code"},{"location":"anomaly_detection/#dsr-a-dual-subspace-re-projection-network-for-surface-anomaly-detection-papers-with-code","text":"https://paperswithcode.com/paper/dsr-a-dual-subspace-re-projection-network-for Proposes an architecture based on quantized feature space representation with dual decoders, DSR, that avoids the image-level anomaly synthesis requirement. Without making any assumptions about the visual properties of anomalies, DSR generates the anomalies at the feature level by sampling the learned quantized feature space, which allows a controlled generation of near-in-distribution anomalies.","title":"DSR -- A dual subspace re-projection network for surface anomaly detection | Papers With Code"},{"location":"ceph/","text":"Manual install of a Ceph Cluster. Daemon container This Dockerfile may be used to bootstrap a Ceph cluster with all the Ceph daemons running. To run a certain type of daemon, simply use the name of the daemon as $1 . Valid values are: mon deploys a Ceph monitor osd deploys an OSD using the method specified by OSD_TYPE osd_directory deploys one or multiple OSDs in a single container using a prepared directory (used in scenario where the operator doesn't want to use --privileged=true ) osd_directory_single deploys an single OSD per container using a prepared directory (used in scenario where the operator doesn't want to use --privileged=true ) osd_ceph_disk deploys an OSD using ceph-disk, so you have to provide a whole device (ie: /dev/sdb) mds deploys a MDS rgw deploys a Rados Gateway Usage You can use this container to bootstrap any Ceph daemon. CLUSTER is the name of the cluster (DEFAULT: ceph) SELinux If SELinux is enabled, run the following commands: sudo chcon -Rt svirt_sandbox_file_t /etc/ceph sudo chcon -Rt svirt_sandbox_file_t /var/lib/ceph KV backends We currently support one KV backend to store our configuration flags, keys and maps: etcd. There is a ceph.defaults config file in the image that is used for defaults to bootstrap daemons. It will add the keys if they are not already present. You can either pre-populate the KV store with your own settings, or provide a ceph.defaults config file. To supply your own defaults, make sure to mount the /etc/ceph/ volume and place your ceph.defaults file there. Important variables in ceph.defaults to add/change when you bootstrap an OSD: /osd/osd_journal_size /osd/cluster_network /osd/public_network Note: cluster_network and public_network are currently not populated in the defaults, but can be passed as environment variables with -e CEPH_PUBLIC_NETWORK=... for more flexibility Populate Key Value store docker run -d --net=host \\ -e KV_TYPE=etcd \\ -e KV_IP=127.0.0.1 \\ -e KV_PORT=2379 \\ ceph/daemon populate_kvstore Zap a device Sometimes you might want to destroy partition tables from a disk. For this you can use the zap_device scenario that works as follow: docker run -d --privileged=true \\ -v /dev/:/dev/ \\ -e OSD_DEVICE=/dev/sdd \\ ceph/daemon zap_device Deploy a monitor A monitor requires some persistent storage for the docker container. If a KV store is used, /etc/ceph will be auto-generated from data kept in the KV store. /var/lib/ceph , however, must be provided by a docker volume. The ceph mon will periodically store data into /var/lib/ceph , including the latest copy of the CRUSH map. If a mon restarts, it will attempt to download the latest monmap and CRUSH map from other peer monitors. However, if all mon daemons have gone down, monitors must be able to recover their previous maps. The docker volume used for /var/lib/ceph should be backed by some durable storage, and must be able to survive container and node restarts. Without KV store, run: docker run -d --net=host \\ -v /etc/ceph:/etc/ceph \\ -v /var/lib/ceph/:/var/lib/ceph/ \\ -e MON_IP=192.168.0.20 \\ -e CEPH_PUBLIC_NETWORK=192.168.0.0/24 \\ ceph/daemon mon With KV store, run: docker run -d --net=host \\ -v /var/lib/ceph:/var/lib/ceph \\ -e MON_IP=192.168.0.20 \\ -e CEPH_PUBLIC_NETWORK=192.168.0.0/24 \\ -e KV_TYPE=etcd \\ -e KV_IP=192.168.0.20 \\ ceph/daemon mon List of available options: MON_NAME : name of the monitor (default to hostname) CEPH_PUBLIC_NETWORK : CIDR of the host running Docker, it should be in the same network as the MON_IP CEPH_CLUSTER_NETWORK : CIDR of a secondary interface of the host running Docker. Used for the OSD replication traffic MON_IP : IP address of the host running Docker NETWORK_AUTO_DETECT : Whether and how to attempt IP and network autodetection. Meant to be used without --net=host . NEW_USER_KEYRING : if specified, it will be imported to keyrings. Works in demo mode only. 0 = Do not detect (default) 1 = Detect IPv6, fallback to IPv4 (if no globally-routable IPv6 address detected) 4 = Detect IPv4 only 6 = Detect IPv6 only Deploy a Manager daemon Since luminous, a manager daemon is mandatory, see docs Without KV store, run: docker run -d --net=host \\ -v /etc/ceph:/etc/ceph \\ -v /var/lib/ceph/:/var/lib/ceph/ \\ ceph/daemon mgr With KV store, run: docker run -d --net=host \\ -v /var/lib/ceph:/var/lib/ceph \\ -e KV_TYPE=etcd \\ -e KV_IP=192.168.0.20 \\ ceph/daemon mgr Deploy an OSD There are four available OSD_TYPE values: - if no OSD_TYPE is set; one of disk , activate or directory will be used based on autodetection of the current OSD bootstrap state activate - the daemon expects to be passed a block device of a ceph-disk -prepared disk (via the OSD_DEVICE environment variable); no bootstrapping will be performed directory - the daemon expects to find the OSD filesystem(s) already mounted in /var/lib/ceph/osd/ disk - the daemon expects to be passed a block device via the OSD_DEVICE environment variable prepare - the daemon expects to be passed a block device and run ceph-disk prepare to bootstrap the disk (via the OSD_DEVICE environment variable) Options for OSDs (TODO: consolidate these options between the types): JOURNAL_DIR - if provided, new OSDs will be bootstrapped to use the specified directory as a common journal area. This is usually used to store the journals for more than one OSD on a common, separate disk. This currently only applies to the directory OSD type. JOURNAL - if provided, the new OSD will be bootstrapped to use the specified journal file (if you do not wish to use the default). This is currently only supported by the directory OSD type OSD_DEVICE - mandatory for activate and disk OSD types; this specifies which block device to use as the OSD OSD_JOURNAL - optional override of the OSD journal file. this only applies to the activate and disk OSD types OSD_FORCE_EXT4 - in case the osd data on ext4 is not automatically recognized (i.e. hidden by overlayfs) you can force them by settings this to yes . Without OSD_TYPE If the operator does not specify an OSD_TYPE autodetection happens: disk is used if no bootstrapped OSD is found. activate is used if a bootstrapped OSD is found and OSD_DEVICE is also provided. directory is used if a bootstrapped OSD is found and no OSD_DEVICE is provided. Without KV backend: docker run -d --net=host \\ --pid=host \\ --privileged=true \\ -v /etc/ceph:/etc/ceph \\ -v /var/lib/ceph/:/var/lib/ceph/ \\ -v /dev/:/dev/ \\ -v /run/udev/:/run/udev/ \\ -e OSD_DEVICE=/dev/vdd \\ ceph/daemon osd With KV backend: docker run -d --net=host \\ --privileged=true \\ --pid=host \\ -v /dev/:/dev/ \\ -v /run/udev/:/run/udev/ \\ -e OSD_DEVICE=/dev/vdd \\ -e KV_TYPE=etcd \\ -e KV_IP=192.168.0.20 \\ ceph/daemon osd Ceph disk Without KV backend: docker run -d --net=host \\ --privileged=true \\ --pid=host \\ -v /etc/ceph:/etc/ceph \\ -v /var/lib/ceph/:/var/lib/ceph/ \\ -v /dev/:/dev/ \\ -v /run/udev/:/run/udev/ \\ -e OSD_DEVICE=/dev/vdd \\ -e OSD_TYPE=disk \\ ceph/daemon osd Using bluestore: docker run -d --net=host \\ --privileged=true \\ --pid=host \\ -v /etc/ceph:/etc/ceph \\ -v /var/lib/ceph/:/var/lib/ceph/ \\ -v /dev/:/dev/ \\ -v /run/udev/:/run/udev/ \\ -e OSD_DEVICE=/dev/vdd \\ -e OSD_TYPE=disk \\ -e OSD_BLUESTORE=1 \\ ceph/daemon osd Using dmcrypt: docker run -d --net=host \\ --privileged=true \\ --pid=host \\ -v /etc/ceph:/etc/ceph \\ -v /var/lib/ceph/:/var/lib/ceph/ \\ -v /dev/:/dev/ \\ -v /run/udev/:/run/udev/ \\ -e OSD_DEVICE=/dev/vdd \\ -e OSD_TYPE=disk \\ -e OSD_DMCRYPT=1 \\ ceph/daemon osd With KV backend: docker run -d --net=host \\ --privileged=true \\ --pid=host \\ -v /dev/:/dev/ \\ -v /run/udev/:/run/udev/ \\ -e OSD_DEVICE=/dev/vdd \\ -e OSD_TYPE=disk \\ -e KV_TYPE=etcd \\ -e KV_IP=192.168.0.20 \\ ceph/daemon osd Using bluestore with KV backend: docker run -d --net=host \\ --privileged=true \\ --pid=host \\ -v /dev/:/dev/ \\ -v /run/udev/:/run/udev/ \\ -e OSD_DEVICE=/dev/vdd \\ -e OSD_TYPE=disk \\ -e KV_TYPE=etcd \\ -e KV_IP=192.168.0.20 \\ -e OSD_BLUESTORE=1 \\ ceph/daemon osd Using dmcrypt with KV backend: docker run -d --net=host \\ --privileged=true \\ --pid=host \\ -v /dev/:/dev/ \\ -v /run/udev/:/run/udev/ \\ -e OSD_DEVICE=/dev/vdd \\ -e OSD_TYPE=disk \\ -e KV_TYPE=etcd \\ -e KV_IP=192.168.0.20 \\ -e OSD_DMCRYPT=1 \\ ceph/daemon osd List of available options: OSD_DEVICE is the OSD device OSD_JOURNAL is the journal for a given OSD HOSTNAME is used to place the OSD in the CRUSH map If you do not want to use --privileged=true , please fall back on the second example. Ceph disk activate This function is balance between ceph-disk and osd directory where the operator can use ceph-disk outside of the container (directly on the host) to prepare the devices. Devices will be prepared with ceph-disk prepare , then they will get activated inside the container. A priviledged container is still required as ceph-disk needs to access /dev/. So this has minimum value compare to the ceph-disk but might fit some use cases where the operators want to prepare their devices outside of a container. docker run -d --net=host \\ --privileged=true \\ --pid=host \\ -v /etc/ceph:/etc/ceph \\ -v /var/lib/ceph/:/var/lib/ceph/ \\ -v /dev/:/dev/ \\ -v /run/udev/:/run/udev/ \\ -e OSD_DEVICE=/dev/vdd \\ -e OSD_TYPE=activate \\ ceph/daemon osd Ceph OSD directory There are a number of environment variables which are used to configure the execution of the OSD: CLUSTER is the name of the ceph cluster (defaults to ceph ) If the OSD is not already created (key, configuration, OSD data), the following environment variables will control its creation: WEIGHT is the of the OSD when it is added to the CRUSH map (default is 1.0 ) JOURNAL is the location of the journal (default is the journal file inside the OSD data directory) HOSTNAME is the name of the host; it is used as a flag when adding the OSD to the CRUSH map The old option OSD_ID is now unused. Instead, the script will scan for each directory in /var/lib/ceph/osd of the form - . To create your OSDs simply run the following command: docker exec ceph osd create . Note that we now default to dropping root privileges, so it is important to set the proper ownership for your OSD directories. The Ceph OSD runs as UID:167, GID:167, so: chown -R 167:167 /var/lib/ceph/osd/ Multiple OSDs There is a problem when attempting run run multiple OSD containers on a single docker host. See issue #19. There are two workarounds, at present: Run each OSD with the --pid=host option Run multiple OSDs within the same container To run multiple OSDs within the same container, simply bind-mount each OSD datastore directory: docker run -v /osds/1:/var/lib/ceph/osd/ceph-1 -v /osds/2:/var/lib/ceph/osd/ceph-2 Ceph OSD directory single Ceph OSD directory single has a similar design to Ceph OSD directory since they both aim to run OSD processes from an already bootstrapped directory. So we assume the OSD directory has been populated already. The major different is that Ceph OSD directory single has a much simpler implementation since it only runs a single OSD process per container. It doesn't do anything with the journal as it assumes journal's symlink was provided during the initialization sequence of the OSD. This scenario goes through the OSD directory ( /var/lib/ceph/osd ) and looks for OSDs that don't have a lock held by any other OSD. If no lock is found, the OSD process starts. If all the OSDs are already running, we gently exit 0 and explain that all the OSDs are already running. Important note : if you are aiming at running multiple OSD containers on a same machine (things that you will likely do with Ceph anyway), you must enable --pid=host . However if you are running Docker 1.12 (based on https://github.com/docker/docker/pull/22481 ), you can just share the same PID namespace for the OSD containers only using: --pid=container: . BTRFS and journal If your OSD is BTRFS and you want to use PARALLEL journal mode, you will need to run this container with --privileged set to true. Otherwise, ceph-osd will have insufficient permissions and it will revert to the slower WRITEAHEAD mode. Note Re: [ https://github.com/Ulexus/docker-ceph/issues/5 ] A user has reported a consterning (and difficult to diagnose) problem wherein the OSD crashes frequently due to Docker running out of sufficient open file handles. This is understandable, as the OSDs use a great many ports during periods of high traffic. It is, therefore, recommended that you increase the number of open file handles available to Docker. On CoreOS (and probably other systemd-based systems), you can do this by creating the a file named /etc/systemd/system/docker.service.d/limits.conf with content something like: [Service] LimitNOFILE=4096 Deploy a MDS By default, the MDS does NOT create a ceph filesystem. If you wish to have this MDS create a ceph filesystem (it will only do this if the specified CEPHFS_NAME does not already exist), you must set, at a minimum, CEPHFS_CREATE=1 . It is strongly recommended that you read the rest of this section, as well. For most people, the defaults for the following optional environment variables are fine, but if you wish to customize the data and metadata pools in which your CephFS is stored, you may override the following as you wish: CEPHFS_CREATE : Whether to create the ceph filesystem (0 = no / 1 = yes), if it doesn't exist. Defaults to 0 (no) CEPHFS_NAME : The name of the new ceph filesystem and the basis on which the later variables are created. Defaults to cephfs CEPHFS_DATA_POOL : The name of the data pool for the ceph filesystem. If it does not exist, it will be created. Defaults to ${CEPHFS_NAME}_data CEPHFS_DATA_POOL_PG : The number of placement groups for the data pool. Defaults to 8 CEPHFS_METADATA_POOL : The name of the metadata pool for the ceph filesystem. If it does not exist, it will be created. Defaults to ${CEPHFS_NAME}_metadata CEPHFS_METADATA_POOL_PG : The number of placement groups for the metadata pool. Defaults to 8 Without KV backend, run: docker run -d --net=host \\ -v /var/lib/ceph/:/var/lib/ceph/ \\ -v /etc/ceph:/etc/ceph \\ -e CEPHFS_CREATE=1 \\ ceph/daemon mds With KV backend, run: docker run -d --net=host \\ -e CEPHFS_CREATE=1 \\ -e KV_TYPE=etcd \\ -e KV_IP=192.168.0.20 \\ ceph/daemon mds List of available options: MDS_NAME is the name the MDS server (DEFAULT: mds-$(hostname)). One thing to note is that metadata servers are not machine-restricted. They are not bound by their data directories and can move around the cluster. As a result, you can run more than one MDS on a single machine. If you plan to do so, you better set this variable and do something like: mds-$(hostname)-a , mds-$(hostname)-b etc... Deploy a Rados Gateway For the Rados Gateway, we deploy it with civetweb enabled by default. However it is possible to use different CGI frontends by simply giving remote address and port. Without kv backend, run: docker run -d --net=host \\ -v /var/lib/ceph/:/var/lib/ceph/ \\ -v /etc/ceph:/etc/ceph \\ ceph/daemon rgw With kv backend, run: docker run -d --net=host \\ -e KV_TYPE=etcd \\ -e KV_IP=192.168.0.20 \\ ceph/daemon rgw List of available options: RGW_CIVETWEB_PORT is the port to which civetweb is listening on (DEFAULT: 8080) RGW_NAME : default to hostname Administration via radosgw-admin from the Docker host if the RGW_NAME variable hasn't been supplied: docker exec radosgw-admin -n client.rgw.$(hostname) -k /var/lib/ceph/radosgw/$(hostname)/keyring If otherwise, $(hostname) has to be replaced by the value of RGW_NAME . To enable an external CGI interface instead of civetweb set: RGW_REMOTE_CGI=1 RGW_REMOTE_CGI_HOST=192.168.0.1 RGW_REMOTE_CGI_PORT=9000 And run the container like this docker run -d -v /etc/ceph:/etc/ceph -v /var/lib/ceph/:/var/lib/ceph -e CEPH_DAEMON=RGW -e RGW_NAME=myrgw -p 9000:9000 -e RGW_REMOTE_CGI=1 -e RGW_REMOTE_CGI_HOST=192.168.0.1 -e RGW_REMOTE_CGI_PORT=9000 ceph/daemon Deploy a REST API This is pretty straightforward. The --net=host is not mandatory, if you don't use it do not forget to expose the RESTAPI_PORT . Only available in luminous. docker run -d --net=host \\ -e KV_TYPE=etcd \\ -e KV_IP=192.168.0.20 \\ ceph/daemon restapi List of available options: RESTAPI_IP is the IP address to listen on (DEFAULT: 0.0.0.0) RESTAPI_PORT is the listening port of the REST API (DEFAULT: 5000) RESTAPI_BASE_URL is the base URL of the API (DEFAULT: /api/v0.1) RESTAPI_LOG_LEVEL is the log level of the API (DEFAULT: warning) RESTAPI_LOG_FILE is the location of the log file (DEFAULT: /var/log/ceph/ceph-restapi.log) Deploy a RBD mirror This is pretty straightforward. The --net=host is not mandatory, with KV we do: docker run -d --net=host \\ -e KV_TYPE=etcd \\ -e KV_IP=192.168.0.20 \\ ceph/daemon rbd_mirror Without KV we do: docker run -d --net=host \\ ceph/daemon rbd_mirror Fetching software. First of I want to check that I have all the latest packages in my debian system. apt update apt upgrade Next we fetch the keys and ceph packages, in this case we download the pacific packages for buster. wget -q -O- 'https://download.ceph.com/keys/release.asc' | sudo apt-key add - echo deb https://download.ceph.com/debian-pacific/ buster main | sudo tee /etc/apt/sources.list.d/ceph.list apt update apt install ceph ceph-common Last we need to download the smartmontools for our nodes. This is so we can monitor our hard drives for hardware issues. echo deb http://deb.debian.org/debian buster-backports main >> /etc/apt/sources.list apt update apt install smartmontools/buster-backports A reboot when you have installed packages is always a good thing and if you need to do some extra hardware changes this is a good place to do so. shutdown -r now Configure node 1 First we will create a ceph configuration file. sudo vi /etc/ceph/ceph.conf The most important things to specify is the id and ips of your cluster monitors. A unique cluster id that you will reuse for all your nodes. And lastly a public network range that you want your monitors to be available over. The cluster network is a good addition if you have the resources to route the recovery traffic on a backbone network. [global] fsid = {cluster uuid} mon initial members = {id1}, {id2}, {id2} mon host = {ip1}, {ip2}, {ip3} public network = {network range for your public network} cluster network = {network range for your cluster network} auth cluster required = cephx auth service required = cephx auth client required = cephx Next we create keys for admin, monitors and boostrapping our drives. These keys will then be merged with the monitor key so the initial setup will have the keys used for other operations. sudo ceph-authtool --create-keyring /tmp/monkey --gen-key -n mon. --cap mon 'allow *' sudo ceph-authtool --create-keyring /etc/ceph/ceph.client.admin.keyring --gen-key -n client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' sudo ceph-authtool --create-keyring /var/lib/ceph/bootstrap-osd/ceph.keyring --gen-key -n client.bootstrap-osd --cap mon 'profile bootstrap-osd' sudo ceph-authtool /tmp/monkey --import-keyring /etc/ceph/ceph.client.admin.keyring sudo ceph-authtool /tmp/monkey --import-keyring /var/lib/ceph/bootstrap-osd/ceph.keyring Make the monitor key available to the ceph user so we don't get an permission error when we start our services. sudo chown ceph:ceph /tmp/monkey Next up we create a monitor map so the monitors will know of each other. The monitors keeps track on other resources but for high availability the monitors needs to know who is in charge. monmaptool --create --add {node1-id} {node1-ip} --fsid {cluster uuid} /tmp/monmap monmaptool --add {node2-id} {node2-ip} --fsid {cluster uuid} /tmp/monmap monmaptool --add {node3-id} {node3-ip} --fsid {cluster uuid} /tmp/monmap Starting a new monitor is as easy as creating a new directory, creating the filesystem for and starting the service. sudo -u ceph mkdir /var/lib/ceph/mon/ceph-{node1-id} sudo -u ceph ceph-mon --mkfs -i {node1-id} --monmap /tmp/monmap --keyring /tmp/monkey sudo systemctl start ceph-mon@{node1-id} Next up we need a manager so we could configure and monitor our cluster through a visual dashboard. First we create a new key, put that key in a newly created directory and start the service. Enabling a dashboard is as easy as running the command for enabling, creating / assigning a certificate and creating a new admin user. sudo ceph auth get-or-create mgr.{node1-id} mon 'allow profile mgr' osd 'allow *' mds 'allow *' sudo -u ceph mkdir /var/lib/ceph/mgr/ceph-{node1-id} sudo -u ceph vi /var/lib/ceph/mgr/ceph-{node1-id}/keyring sudo systemctl start ceph-mgr@{node1-id} sudo ceph mgr module enable dashboard sudo ceph dashboard create-self-signed-cert sudo ceph dashboard ac-user-create admin -i passwd administrator Setting up more nodes. First of we need to copy over the configuration, monitor map and all the keys over to our new host. sudo scp {user}@{server}:/etc/ceph/ceph.conf /etc/ceph/ceph.conf sudo scp {user}@{server}:/etc/ceph/ceph.client.admin.keyring /etc/ceph/ceph.client.admin.keyring sudo scp {user}@{server}:/var/lib/ceph/bootstrap-osd/ceph.keyring /var/lib/ceph/bootstrap-osd/ceph.keyring sudo scp {user}@{server}:/tmp/monmap /tmp/monmap sudo scp {user}@{server}:/tmp/monkey /tmp/monkey Next up we setup the monitor node exactly as we did with the first node. sudo -u ceph mkdir /var/lib/ceph/mon/ceph-{node2-id} sudo -u ceph ceph-mon --mkfs -i {node2-id} --monmap /tmp/monmap --keyring /tmp/monkey sudo systemctl start ceph-mon@{node2-id} sudo ceph -s sudo ceph mon enable-msgr2 Then we setup the manager node exactly as we did with the first node. sudo ceph auth get-or-create mgr.{node2-id} mon 'allow profile mgr' osd 'allow *' mds 'allow *' sudo -u ceph mkdir /var/lib/ceph/mgr/ceph-{node2-id} sudo -u ceph vi /var/lib/ceph/mgr/ceph-{node2-id}/keyring sudo systemctl start ceph-mgr@{node2-id} Adding storage When the cluster is up and running and all monitors are in qourum you could add storage services. This is easily done via the volume command. First prepare a disk so it will be known by the cluster and have the keys and configuration copied to the management directory. Next up you activate the service so your storage nodes will be ready to use. This will be done for all the harddrives you want to add to your network. sudo ceph-volume lvm prepare --data /dev/sdb sudo ceph-volume lvm activate {osd-number} {osd-uuid} Post configuration Last but not least you want to ensure that all the services starts after a reboot. In debian you do that by enabling the services. sudo systemctl enable ceph-mon@{node-id} sudo systemctl enable ceph-mgr@{node-id} sudo systemctl enable ceph-osd@{osd-number}","title":"ceph"},{"location":"ceph/#manual-install-of-a-ceph-cluster","text":"","title":"Manual install of a Ceph Cluster."},{"location":"ceph/#daemon-container","text":"This Dockerfile may be used to bootstrap a Ceph cluster with all the Ceph daemons running. To run a certain type of daemon, simply use the name of the daemon as $1 . Valid values are: mon deploys a Ceph monitor osd deploys an OSD using the method specified by OSD_TYPE osd_directory deploys one or multiple OSDs in a single container using a prepared directory (used in scenario where the operator doesn't want to use --privileged=true ) osd_directory_single deploys an single OSD per container using a prepared directory (used in scenario where the operator doesn't want to use --privileged=true ) osd_ceph_disk deploys an OSD using ceph-disk, so you have to provide a whole device (ie: /dev/sdb) mds deploys a MDS rgw deploys a Rados Gateway","title":"Daemon container"},{"location":"ceph/#usage","text":"You can use this container to bootstrap any Ceph daemon. CLUSTER is the name of the cluster (DEFAULT: ceph)","title":"Usage"},{"location":"ceph/#selinux","text":"If SELinux is enabled, run the following commands: sudo chcon -Rt svirt_sandbox_file_t /etc/ceph sudo chcon -Rt svirt_sandbox_file_t /var/lib/ceph","title":"SELinux"},{"location":"ceph/#kv-backends","text":"We currently support one KV backend to store our configuration flags, keys and maps: etcd. There is a ceph.defaults config file in the image that is used for defaults to bootstrap daemons. It will add the keys if they are not already present. You can either pre-populate the KV store with your own settings, or provide a ceph.defaults config file. To supply your own defaults, make sure to mount the /etc/ceph/ volume and place your ceph.defaults file there. Important variables in ceph.defaults to add/change when you bootstrap an OSD: /osd/osd_journal_size /osd/cluster_network /osd/public_network Note: cluster_network and public_network are currently not populated in the defaults, but can be passed as environment variables with -e CEPH_PUBLIC_NETWORK=... for more flexibility","title":"KV backends"},{"location":"ceph/#populate-key-value-store","text":"docker run -d --net=host \\ -e KV_TYPE=etcd \\ -e KV_IP=127.0.0.1 \\ -e KV_PORT=2379 \\ ceph/daemon populate_kvstore","title":"Populate Key Value store"},{"location":"ceph/#zap-a-device","text":"Sometimes you might want to destroy partition tables from a disk. For this you can use the zap_device scenario that works as follow: docker run -d --privileged=true \\ -v /dev/:/dev/ \\ -e OSD_DEVICE=/dev/sdd \\ ceph/daemon zap_device","title":"Zap a device"},{"location":"ceph/#deploy-a-monitor","text":"A monitor requires some persistent storage for the docker container. If a KV store is used, /etc/ceph will be auto-generated from data kept in the KV store. /var/lib/ceph , however, must be provided by a docker volume. The ceph mon will periodically store data into /var/lib/ceph , including the latest copy of the CRUSH map. If a mon restarts, it will attempt to download the latest monmap and CRUSH map from other peer monitors. However, if all mon daemons have gone down, monitors must be able to recover their previous maps. The docker volume used for /var/lib/ceph should be backed by some durable storage, and must be able to survive container and node restarts. Without KV store, run: docker run -d --net=host \\ -v /etc/ceph:/etc/ceph \\ -v /var/lib/ceph/:/var/lib/ceph/ \\ -e MON_IP=192.168.0.20 \\ -e CEPH_PUBLIC_NETWORK=192.168.0.0/24 \\ ceph/daemon mon With KV store, run: docker run -d --net=host \\ -v /var/lib/ceph:/var/lib/ceph \\ -e MON_IP=192.168.0.20 \\ -e CEPH_PUBLIC_NETWORK=192.168.0.0/24 \\ -e KV_TYPE=etcd \\ -e KV_IP=192.168.0.20 \\ ceph/daemon mon List of available options: MON_NAME : name of the monitor (default to hostname) CEPH_PUBLIC_NETWORK : CIDR of the host running Docker, it should be in the same network as the MON_IP CEPH_CLUSTER_NETWORK : CIDR of a secondary interface of the host running Docker. Used for the OSD replication traffic MON_IP : IP address of the host running Docker NETWORK_AUTO_DETECT : Whether and how to attempt IP and network autodetection. Meant to be used without --net=host . NEW_USER_KEYRING : if specified, it will be imported to keyrings. Works in demo mode only. 0 = Do not detect (default) 1 = Detect IPv6, fallback to IPv4 (if no globally-routable IPv6 address detected) 4 = Detect IPv4 only 6 = Detect IPv6 only","title":"Deploy a monitor"},{"location":"ceph/#deploy-a-manager-daemon","text":"Since luminous, a manager daemon is mandatory, see docs Without KV store, run: docker run -d --net=host \\ -v /etc/ceph:/etc/ceph \\ -v /var/lib/ceph/:/var/lib/ceph/ \\ ceph/daemon mgr With KV store, run: docker run -d --net=host \\ -v /var/lib/ceph:/var/lib/ceph \\ -e KV_TYPE=etcd \\ -e KV_IP=192.168.0.20 \\ ceph/daemon mgr","title":"Deploy a Manager daemon"},{"location":"ceph/#deploy-an-osd","text":"There are four available OSD_TYPE values: - if no OSD_TYPE is set; one of disk , activate or directory will be used based on autodetection of the current OSD bootstrap state activate - the daemon expects to be passed a block device of a ceph-disk -prepared disk (via the OSD_DEVICE environment variable); no bootstrapping will be performed directory - the daemon expects to find the OSD filesystem(s) already mounted in /var/lib/ceph/osd/ disk - the daemon expects to be passed a block device via the OSD_DEVICE environment variable prepare - the daemon expects to be passed a block device and run ceph-disk prepare to bootstrap the disk (via the OSD_DEVICE environment variable) Options for OSDs (TODO: consolidate these options between the types): JOURNAL_DIR - if provided, new OSDs will be bootstrapped to use the specified directory as a common journal area. This is usually used to store the journals for more than one OSD on a common, separate disk. This currently only applies to the directory OSD type. JOURNAL - if provided, the new OSD will be bootstrapped to use the specified journal file (if you do not wish to use the default). This is currently only supported by the directory OSD type OSD_DEVICE - mandatory for activate and disk OSD types; this specifies which block device to use as the OSD OSD_JOURNAL - optional override of the OSD journal file. this only applies to the activate and disk OSD types OSD_FORCE_EXT4 - in case the osd data on ext4 is not automatically recognized (i.e. hidden by overlayfs) you can force them by settings this to yes .","title":"Deploy an OSD"},{"location":"ceph/#without-osd_type","text":"If the operator does not specify an OSD_TYPE autodetection happens: disk is used if no bootstrapped OSD is found. activate is used if a bootstrapped OSD is found and OSD_DEVICE is also provided. directory is used if a bootstrapped OSD is found and no OSD_DEVICE is provided. Without KV backend: docker run -d --net=host \\ --pid=host \\ --privileged=true \\ -v /etc/ceph:/etc/ceph \\ -v /var/lib/ceph/:/var/lib/ceph/ \\ -v /dev/:/dev/ \\ -v /run/udev/:/run/udev/ \\ -e OSD_DEVICE=/dev/vdd \\ ceph/daemon osd With KV backend: docker run -d --net=host \\ --privileged=true \\ --pid=host \\ -v /dev/:/dev/ \\ -v /run/udev/:/run/udev/ \\ -e OSD_DEVICE=/dev/vdd \\ -e KV_TYPE=etcd \\ -e KV_IP=192.168.0.20 \\ ceph/daemon osd","title":"Without OSD_TYPE"},{"location":"ceph/#ceph-disk","text":"Without KV backend: docker run -d --net=host \\ --privileged=true \\ --pid=host \\ -v /etc/ceph:/etc/ceph \\ -v /var/lib/ceph/:/var/lib/ceph/ \\ -v /dev/:/dev/ \\ -v /run/udev/:/run/udev/ \\ -e OSD_DEVICE=/dev/vdd \\ -e OSD_TYPE=disk \\ ceph/daemon osd Using bluestore: docker run -d --net=host \\ --privileged=true \\ --pid=host \\ -v /etc/ceph:/etc/ceph \\ -v /var/lib/ceph/:/var/lib/ceph/ \\ -v /dev/:/dev/ \\ -v /run/udev/:/run/udev/ \\ -e OSD_DEVICE=/dev/vdd \\ -e OSD_TYPE=disk \\ -e OSD_BLUESTORE=1 \\ ceph/daemon osd Using dmcrypt: docker run -d --net=host \\ --privileged=true \\ --pid=host \\ -v /etc/ceph:/etc/ceph \\ -v /var/lib/ceph/:/var/lib/ceph/ \\ -v /dev/:/dev/ \\ -v /run/udev/:/run/udev/ \\ -e OSD_DEVICE=/dev/vdd \\ -e OSD_TYPE=disk \\ -e OSD_DMCRYPT=1 \\ ceph/daemon osd With KV backend: docker run -d --net=host \\ --privileged=true \\ --pid=host \\ -v /dev/:/dev/ \\ -v /run/udev/:/run/udev/ \\ -e OSD_DEVICE=/dev/vdd \\ -e OSD_TYPE=disk \\ -e KV_TYPE=etcd \\ -e KV_IP=192.168.0.20 \\ ceph/daemon osd Using bluestore with KV backend: docker run -d --net=host \\ --privileged=true \\ --pid=host \\ -v /dev/:/dev/ \\ -v /run/udev/:/run/udev/ \\ -e OSD_DEVICE=/dev/vdd \\ -e OSD_TYPE=disk \\ -e KV_TYPE=etcd \\ -e KV_IP=192.168.0.20 \\ -e OSD_BLUESTORE=1 \\ ceph/daemon osd Using dmcrypt with KV backend: docker run -d --net=host \\ --privileged=true \\ --pid=host \\ -v /dev/:/dev/ \\ -v /run/udev/:/run/udev/ \\ -e OSD_DEVICE=/dev/vdd \\ -e OSD_TYPE=disk \\ -e KV_TYPE=etcd \\ -e KV_IP=192.168.0.20 \\ -e OSD_DMCRYPT=1 \\ ceph/daemon osd List of available options: OSD_DEVICE is the OSD device OSD_JOURNAL is the journal for a given OSD HOSTNAME is used to place the OSD in the CRUSH map If you do not want to use --privileged=true , please fall back on the second example.","title":"Ceph disk"},{"location":"ceph/#ceph-disk-activate","text":"This function is balance between ceph-disk and osd directory where the operator can use ceph-disk outside of the container (directly on the host) to prepare the devices. Devices will be prepared with ceph-disk prepare , then they will get activated inside the container. A priviledged container is still required as ceph-disk needs to access /dev/. So this has minimum value compare to the ceph-disk but might fit some use cases where the operators want to prepare their devices outside of a container. docker run -d --net=host \\ --privileged=true \\ --pid=host \\ -v /etc/ceph:/etc/ceph \\ -v /var/lib/ceph/:/var/lib/ceph/ \\ -v /dev/:/dev/ \\ -v /run/udev/:/run/udev/ \\ -e OSD_DEVICE=/dev/vdd \\ -e OSD_TYPE=activate \\ ceph/daemon osd","title":"Ceph disk activate"},{"location":"ceph/#ceph-osd-directory","text":"There are a number of environment variables which are used to configure the execution of the OSD: CLUSTER is the name of the ceph cluster (defaults to ceph ) If the OSD is not already created (key, configuration, OSD data), the following environment variables will control its creation: WEIGHT is the of the OSD when it is added to the CRUSH map (default is 1.0 ) JOURNAL is the location of the journal (default is the journal file inside the OSD data directory) HOSTNAME is the name of the host; it is used as a flag when adding the OSD to the CRUSH map The old option OSD_ID is now unused. Instead, the script will scan for each directory in /var/lib/ceph/osd of the form - . To create your OSDs simply run the following command: docker exec ceph osd create . Note that we now default to dropping root privileges, so it is important to set the proper ownership for your OSD directories. The Ceph OSD runs as UID:167, GID:167, so: chown -R 167:167 /var/lib/ceph/osd/","title":"Ceph OSD directory"},{"location":"ceph/#multiple-osds","text":"There is a problem when attempting run run multiple OSD containers on a single docker host. See issue #19. There are two workarounds, at present: Run each OSD with the --pid=host option Run multiple OSDs within the same container To run multiple OSDs within the same container, simply bind-mount each OSD datastore directory: docker run -v /osds/1:/var/lib/ceph/osd/ceph-1 -v /osds/2:/var/lib/ceph/osd/ceph-2","title":"Multiple OSDs"},{"location":"ceph/#ceph-osd-directory-single","text":"Ceph OSD directory single has a similar design to Ceph OSD directory since they both aim to run OSD processes from an already bootstrapped directory. So we assume the OSD directory has been populated already. The major different is that Ceph OSD directory single has a much simpler implementation since it only runs a single OSD process per container. It doesn't do anything with the journal as it assumes journal's symlink was provided during the initialization sequence of the OSD. This scenario goes through the OSD directory ( /var/lib/ceph/osd ) and looks for OSDs that don't have a lock held by any other OSD. If no lock is found, the OSD process starts. If all the OSDs are already running, we gently exit 0 and explain that all the OSDs are already running. Important note : if you are aiming at running multiple OSD containers on a same machine (things that you will likely do with Ceph anyway), you must enable --pid=host . However if you are running Docker 1.12 (based on https://github.com/docker/docker/pull/22481 ), you can just share the same PID namespace for the OSD containers only using: --pid=container: .","title":"Ceph OSD directory single"},{"location":"ceph/#btrfs-and-journal","text":"If your OSD is BTRFS and you want to use PARALLEL journal mode, you will need to run this container with --privileged set to true. Otherwise, ceph-osd will have insufficient permissions and it will revert to the slower WRITEAHEAD mode.","title":"BTRFS and journal"},{"location":"ceph/#note","text":"Re: [ https://github.com/Ulexus/docker-ceph/issues/5 ] A user has reported a consterning (and difficult to diagnose) problem wherein the OSD crashes frequently due to Docker running out of sufficient open file handles. This is understandable, as the OSDs use a great many ports during periods of high traffic. It is, therefore, recommended that you increase the number of open file handles available to Docker. On CoreOS (and probably other systemd-based systems), you can do this by creating the a file named /etc/systemd/system/docker.service.d/limits.conf with content something like: [Service] LimitNOFILE=4096","title":"Note"},{"location":"ceph/#deploy-a-mds","text":"By default, the MDS does NOT create a ceph filesystem. If you wish to have this MDS create a ceph filesystem (it will only do this if the specified CEPHFS_NAME does not already exist), you must set, at a minimum, CEPHFS_CREATE=1 . It is strongly recommended that you read the rest of this section, as well. For most people, the defaults for the following optional environment variables are fine, but if you wish to customize the data and metadata pools in which your CephFS is stored, you may override the following as you wish: CEPHFS_CREATE : Whether to create the ceph filesystem (0 = no / 1 = yes), if it doesn't exist. Defaults to 0 (no) CEPHFS_NAME : The name of the new ceph filesystem and the basis on which the later variables are created. Defaults to cephfs CEPHFS_DATA_POOL : The name of the data pool for the ceph filesystem. If it does not exist, it will be created. Defaults to ${CEPHFS_NAME}_data CEPHFS_DATA_POOL_PG : The number of placement groups for the data pool. Defaults to 8 CEPHFS_METADATA_POOL : The name of the metadata pool for the ceph filesystem. If it does not exist, it will be created. Defaults to ${CEPHFS_NAME}_metadata CEPHFS_METADATA_POOL_PG : The number of placement groups for the metadata pool. Defaults to 8 Without KV backend, run: docker run -d --net=host \\ -v /var/lib/ceph/:/var/lib/ceph/ \\ -v /etc/ceph:/etc/ceph \\ -e CEPHFS_CREATE=1 \\ ceph/daemon mds With KV backend, run: docker run -d --net=host \\ -e CEPHFS_CREATE=1 \\ -e KV_TYPE=etcd \\ -e KV_IP=192.168.0.20 \\ ceph/daemon mds List of available options: MDS_NAME is the name the MDS server (DEFAULT: mds-$(hostname)). One thing to note is that metadata servers are not machine-restricted. They are not bound by their data directories and can move around the cluster. As a result, you can run more than one MDS on a single machine. If you plan to do so, you better set this variable and do something like: mds-$(hostname)-a , mds-$(hostname)-b etc...","title":"Deploy a MDS"},{"location":"ceph/#deploy-a-rados-gateway","text":"For the Rados Gateway, we deploy it with civetweb enabled by default. However it is possible to use different CGI frontends by simply giving remote address and port. Without kv backend, run: docker run -d --net=host \\ -v /var/lib/ceph/:/var/lib/ceph/ \\ -v /etc/ceph:/etc/ceph \\ ceph/daemon rgw With kv backend, run: docker run -d --net=host \\ -e KV_TYPE=etcd \\ -e KV_IP=192.168.0.20 \\ ceph/daemon rgw List of available options: RGW_CIVETWEB_PORT is the port to which civetweb is listening on (DEFAULT: 8080) RGW_NAME : default to hostname Administration via radosgw-admin from the Docker host if the RGW_NAME variable hasn't been supplied: docker exec radosgw-admin -n client.rgw.$(hostname) -k /var/lib/ceph/radosgw/$(hostname)/keyring If otherwise, $(hostname) has to be replaced by the value of RGW_NAME . To enable an external CGI interface instead of civetweb set: RGW_REMOTE_CGI=1 RGW_REMOTE_CGI_HOST=192.168.0.1 RGW_REMOTE_CGI_PORT=9000 And run the container like this docker run -d -v /etc/ceph:/etc/ceph -v /var/lib/ceph/:/var/lib/ceph -e CEPH_DAEMON=RGW -e RGW_NAME=myrgw -p 9000:9000 -e RGW_REMOTE_CGI=1 -e RGW_REMOTE_CGI_HOST=192.168.0.1 -e RGW_REMOTE_CGI_PORT=9000 ceph/daemon","title":"Deploy a Rados Gateway"},{"location":"ceph/#deploy-a-rest-api","text":"This is pretty straightforward. The --net=host is not mandatory, if you don't use it do not forget to expose the RESTAPI_PORT . Only available in luminous. docker run -d --net=host \\ -e KV_TYPE=etcd \\ -e KV_IP=192.168.0.20 \\ ceph/daemon restapi List of available options: RESTAPI_IP is the IP address to listen on (DEFAULT: 0.0.0.0) RESTAPI_PORT is the listening port of the REST API (DEFAULT: 5000) RESTAPI_BASE_URL is the base URL of the API (DEFAULT: /api/v0.1) RESTAPI_LOG_LEVEL is the log level of the API (DEFAULT: warning) RESTAPI_LOG_FILE is the location of the log file (DEFAULT: /var/log/ceph/ceph-restapi.log)","title":"Deploy a REST API"},{"location":"ceph/#deploy-a-rbd-mirror","text":"This is pretty straightforward. The --net=host is not mandatory, with KV we do: docker run -d --net=host \\ -e KV_TYPE=etcd \\ -e KV_IP=192.168.0.20 \\ ceph/daemon rbd_mirror Without KV we do: docker run -d --net=host \\ ceph/daemon rbd_mirror","title":"Deploy a RBD mirror"},{"location":"ceph/#fetching-software","text":"First of I want to check that I have all the latest packages in my debian system. apt update apt upgrade Next we fetch the keys and ceph packages, in this case we download the pacific packages for buster. wget -q -O- 'https://download.ceph.com/keys/release.asc' | sudo apt-key add - echo deb https://download.ceph.com/debian-pacific/ buster main | sudo tee /etc/apt/sources.list.d/ceph.list apt update apt install ceph ceph-common Last we need to download the smartmontools for our nodes. This is so we can monitor our hard drives for hardware issues. echo deb http://deb.debian.org/debian buster-backports main >> /etc/apt/sources.list apt update apt install smartmontools/buster-backports A reboot when you have installed packages is always a good thing and if you need to do some extra hardware changes this is a good place to do so. shutdown -r now","title":"Fetching software."},{"location":"ceph/#configure-node-1","text":"First we will create a ceph configuration file. sudo vi /etc/ceph/ceph.conf The most important things to specify is the id and ips of your cluster monitors. A unique cluster id that you will reuse for all your nodes. And lastly a public network range that you want your monitors to be available over. The cluster network is a good addition if you have the resources to route the recovery traffic on a backbone network. [global] fsid = {cluster uuid} mon initial members = {id1}, {id2}, {id2} mon host = {ip1}, {ip2}, {ip3} public network = {network range for your public network} cluster network = {network range for your cluster network} auth cluster required = cephx auth service required = cephx auth client required = cephx Next we create keys for admin, monitors and boostrapping our drives. These keys will then be merged with the monitor key so the initial setup will have the keys used for other operations. sudo ceph-authtool --create-keyring /tmp/monkey --gen-key -n mon. --cap mon 'allow *' sudo ceph-authtool --create-keyring /etc/ceph/ceph.client.admin.keyring --gen-key -n client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' sudo ceph-authtool --create-keyring /var/lib/ceph/bootstrap-osd/ceph.keyring --gen-key -n client.bootstrap-osd --cap mon 'profile bootstrap-osd' sudo ceph-authtool /tmp/monkey --import-keyring /etc/ceph/ceph.client.admin.keyring sudo ceph-authtool /tmp/monkey --import-keyring /var/lib/ceph/bootstrap-osd/ceph.keyring Make the monitor key available to the ceph user so we don't get an permission error when we start our services. sudo chown ceph:ceph /tmp/monkey Next up we create a monitor map so the monitors will know of each other. The monitors keeps track on other resources but for high availability the monitors needs to know who is in charge. monmaptool --create --add {node1-id} {node1-ip} --fsid {cluster uuid} /tmp/monmap monmaptool --add {node2-id} {node2-ip} --fsid {cluster uuid} /tmp/monmap monmaptool --add {node3-id} {node3-ip} --fsid {cluster uuid} /tmp/monmap Starting a new monitor is as easy as creating a new directory, creating the filesystem for and starting the service. sudo -u ceph mkdir /var/lib/ceph/mon/ceph-{node1-id} sudo -u ceph ceph-mon --mkfs -i {node1-id} --monmap /tmp/monmap --keyring /tmp/monkey sudo systemctl start ceph-mon@{node1-id} Next up we need a manager so we could configure and monitor our cluster through a visual dashboard. First we create a new key, put that key in a newly created directory and start the service. Enabling a dashboard is as easy as running the command for enabling, creating / assigning a certificate and creating a new admin user. sudo ceph auth get-or-create mgr.{node1-id} mon 'allow profile mgr' osd 'allow *' mds 'allow *' sudo -u ceph mkdir /var/lib/ceph/mgr/ceph-{node1-id} sudo -u ceph vi /var/lib/ceph/mgr/ceph-{node1-id}/keyring sudo systemctl start ceph-mgr@{node1-id} sudo ceph mgr module enable dashboard sudo ceph dashboard create-self-signed-cert sudo ceph dashboard ac-user-create admin -i passwd administrator","title":"Configure node 1"},{"location":"ceph/#setting-up-more-nodes","text":"First of we need to copy over the configuration, monitor map and all the keys over to our new host. sudo scp {user}@{server}:/etc/ceph/ceph.conf /etc/ceph/ceph.conf sudo scp {user}@{server}:/etc/ceph/ceph.client.admin.keyring /etc/ceph/ceph.client.admin.keyring sudo scp {user}@{server}:/var/lib/ceph/bootstrap-osd/ceph.keyring /var/lib/ceph/bootstrap-osd/ceph.keyring sudo scp {user}@{server}:/tmp/monmap /tmp/monmap sudo scp {user}@{server}:/tmp/monkey /tmp/monkey Next up we setup the monitor node exactly as we did with the first node. sudo -u ceph mkdir /var/lib/ceph/mon/ceph-{node2-id} sudo -u ceph ceph-mon --mkfs -i {node2-id} --monmap /tmp/monmap --keyring /tmp/monkey sudo systemctl start ceph-mon@{node2-id} sudo ceph -s sudo ceph mon enable-msgr2 Then we setup the manager node exactly as we did with the first node. sudo ceph auth get-or-create mgr.{node2-id} mon 'allow profile mgr' osd 'allow *' mds 'allow *' sudo -u ceph mkdir /var/lib/ceph/mgr/ceph-{node2-id} sudo -u ceph vi /var/lib/ceph/mgr/ceph-{node2-id}/keyring sudo systemctl start ceph-mgr@{node2-id}","title":"Setting up more nodes."},{"location":"ceph/#adding-storage","text":"When the cluster is up and running and all monitors are in qourum you could add storage services. This is easily done via the volume command. First prepare a disk so it will be known by the cluster and have the keys and configuration copied to the management directory. Next up you activate the service so your storage nodes will be ready to use. This will be done for all the harddrives you want to add to your network. sudo ceph-volume lvm prepare --data /dev/sdb sudo ceph-volume lvm activate {osd-number} {osd-uuid}","title":"Adding storage"},{"location":"ceph/#post-configuration","text":"Last but not least you want to ensure that all the services starts after a reboot. In debian you do that by enabling the services. sudo systemctl enable ceph-mon@{node-id} sudo systemctl enable ceph-mgr@{node-id} sudo systemctl enable ceph-osd@{osd-number}","title":"Post configuration"},{"location":"chart/","text":"chart","title":"Chart"},{"location":"chart/#chart","text":"","title":"chart"},{"location":"code/","text":"code install vscode git clone https://AUR.archlinux.org/visual-studio-code-bin.git cd visual-studio-code-bin makepkg -s sudo pacman -U *code-bin-*.pkg.tar.zst cd ../ && sudo rm -rfv visual-studio-code-bin/","title":"vscode"},{"location":"code/#code","text":"","title":"code"},{"location":"code/#install-vscode","text":"git clone https://AUR.archlinux.org/visual-studio-code-bin.git cd visual-studio-code-bin makepkg -s sudo pacman -U *code-bin-*.pkg.tar.zst cd ../ && sudo rm -rfv visual-studio-code-bin/","title":"install vscode"},{"location":"conda/","text":"conda cheat sheet conda update -n base conda conda remove --name myenv --all conda env remove -n tf2 conda init powershell conda env create -f environment.yml conda env export > environment.yml conda create --name myenv python=3.10 conda install --file requirements.txt conda install --file requirements.txt -c conda-forge conda create --name tft --clone 311 pip install -r /path/to/requirements.txt pip install --upgrade --force-reinstall -r requirements.txt python -m ipykernel install --user --name myenv --display-name \"Python (myenv)\" conda clean -a conda install -y -c apple tensorflow-deps pip install tensorflow-macos tensorflow-metal bash Miniconda3-py310_22.11.1-1-Linux-x86_64.sh conda list conda config --add channels conda-forge conda config --describe conda config --set auto_activate_base false conda update --all pip --disable-pip-version-check list --outdated --format=json | python -c \"import json, sys; print('\\n'.join([x['name'] for x in json.load(sys.stdin)]))\" | xargs -n1 pip install -U conda update conda conda install --file requirements.txt -c conda-forge pip install -r requirements.txt # archlinux echo \"[ -f /opt/miniconda3/etc/profile.d/conda.sh ] && source /opt/miniconda3/etc/profile.d/conda.sh\" >> ~/.zshrc # ubuntu mkdir -p ~/miniconda3 wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh -O ~/miniconda3/miniconda.sh bash ~/miniconda3/miniconda.sh -b -u -p ~/miniconda3 rm -rf ~/miniconda3/miniconda.sh source ~/miniconda3/etc/profile.d/conda.sh conda init zsh","title":"Conda"},{"location":"conda/#conda-cheat-sheet","text":"conda update -n base conda conda remove --name myenv --all conda env remove -n tf2 conda init powershell conda env create -f environment.yml conda env export > environment.yml conda create --name myenv python=3.10 conda install --file requirements.txt conda install --file requirements.txt -c conda-forge conda create --name tft --clone 311 pip install -r /path/to/requirements.txt pip install --upgrade --force-reinstall -r requirements.txt python -m ipykernel install --user --name myenv --display-name \"Python (myenv)\" conda clean -a conda install -y -c apple tensorflow-deps pip install tensorflow-macos tensorflow-metal bash Miniconda3-py310_22.11.1-1-Linux-x86_64.sh conda list conda config --add channels conda-forge conda config --describe conda config --set auto_activate_base false conda update --all pip --disable-pip-version-check list --outdated --format=json | python -c \"import json, sys; print('\\n'.join([x['name'] for x in json.load(sys.stdin)]))\" | xargs -n1 pip install -U conda update conda conda install --file requirements.txt -c conda-forge pip install -r requirements.txt # archlinux echo \"[ -f /opt/miniconda3/etc/profile.d/conda.sh ] && source /opt/miniconda3/etc/profile.d/conda.sh\" >> ~/.zshrc # ubuntu mkdir -p ~/miniconda3 wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh -O ~/miniconda3/miniconda.sh bash ~/miniconda3/miniconda.sh -b -u -p ~/miniconda3 rm -rf ~/miniconda3/miniconda.sh source ~/miniconda3/etc/profile.d/conda.sh conda init zsh","title":"conda cheat sheet"},{"location":"dataframe/","text":"Dataframe df[\"cancer\"].value_counts().plot(kind='barh') df[[\"patient_id\", \"image_id\"]].astype(str).apply(lambda x: 'data/rsna/'+'/'.join(x)+'.png', axis=1) list(mapping.keys())[list(mapping.values()).index(x)] `` import logging from configs import LOGGER_NAME logger = logging.getLogger(LOGGER_NAME) # pylint: disable=invalid-name logger.info Rows of a pandas dataframe df def __len__(self) -> int: \"\"\"Return the length of the dataset.\"\"\" return len(self.df.index) In [7]: timeit len(df.index) 1000000 loops, best of 3: 248 ns per loop In [8]: timeit len(df) 1000000 loops, best of 3: 573 ns per loop","title":"Dataframe"},{"location":"dataframe/#dataframe","text":"df[\"cancer\"].value_counts().plot(kind='barh') df[[\"patient_id\", \"image_id\"]].astype(str).apply(lambda x: 'data/rsna/'+'/'.join(x)+'.png', axis=1) list(mapping.keys())[list(mapping.values()).index(x)] `` import logging from configs import LOGGER_NAME logger = logging.getLogger(LOGGER_NAME) # pylint: disable=invalid-name logger.info","title":"Dataframe"},{"location":"dataframe/#rows-of-a-pandas-dataframe-df","text":"def __len__(self) -> int: \"\"\"Return the length of the dataset.\"\"\" return len(self.df.index) In [7]: timeit len(df.index) 1000000 loops, best of 3: 248 ns per loop In [8]: timeit len(df) 1000000 loops, best of 3: 573 ns per loop","title":"Rows of a pandas dataframe df"},{"location":"db/","text":"SQLAlchemy conda create --name sql python=3.11 pip install SQLAlchemy pip install mariadb pip install PyMySQL brew install mariadb-connector-c brew install mysql docker-compose up --build -d","title":"database"},{"location":"db/#sqlalchemy","text":"conda create --name sql python=3.11 pip install SQLAlchemy pip install mariadb pip install PyMySQL brew install mariadb-connector-c brew install mysql docker-compose up --build -d","title":"SQLAlchemy"},{"location":"docker/","text":"Docker sudo apt update && sudo apt upgrade -y sudo apt install git curl htop mc rsync zsh-autosuggestions sudo apt install zsh chsh -s $(which zsh) sh -c \"$(curl -fsSL https://raw.githubusercontent.com/ohmyzsh/ohmyzsh/master/tools/install.sh)\" rsync -avuPz ./.oh-my-zsh/custom/themes/custom.zsh-theme furyhawk@arm:/home/furyhawk/.oh-my-zsh/custom/themes/custom.zsh-theme git clone https://github.com/zsh-users/zsh-autosuggestions.git $ZSH_CUSTOM/plugins/zsh-autosuggestions git clone https://github.com/zdharma-continuum/fast-syntax-highlighting.git ${ZSH_CUSTOM:-$HOME/.oh-my-zsh/custom}/plugins/fast-syntax-highlighting ## Enable plugins by adding them to .zshrc. - Open .zshrc `nano ~/.zshrc` - ZSH_THEME=\"custom\" - Find the line which says `plugins=(git)`. - Replace that line with `plugins=(git zsh-autosuggestions zsh-syntax-highlighting fast-syntax-highlighting zsh-autocomplete)` touch .zprofile nano .zprofile # set PATH so it includes user's private bin if it exists if [ -d \"$HOME/bin\" ] ; then PATH=\"$HOME/bin:$PATH\" fi # set PATH so it includes user's private bin if it exists if [ -d \"$HOME/.local/bin\" ] ; then PATH=\"$HOME/.local/bin:$PATH\" fi export TZ=Asia/Singapore export NODE_ID=$(docker info -f '{{.Swarm.NodeID}}') export EMAIL=furyx@hotmail.com export DOMAIN=furyhawk.lol sudo nano /etc/fstab touch .credentials nano .credentials sudo curl -fsSL https://download.docker.com/linux/debian/gpg -o /etc/apt/keyrings/docker.asc sudo chmod a+r /etc/apt/keyrings/docker.asc echo \\ \"deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.asc] https://download.docker.com/linux/debian \\ $(. /etc/os-release && echo \"$VERSION_CODENAME\") stable\" | \\ sudo tee /etc/apt/sources.list.d/docker.list > /dev/null sudo apt-get update sudo apt-get install docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin # sudo groupadd docker sudo usermod -aG docker $USER sudo systemctl enable docker.service sudo systemctl enable containerd.service systemctl list-units --type=service --state=active cd /etc/docker sudo touch daemon.json sudo nano daemon.json { \"log-driver\": \"json-file\", \"log-opts\": { \"max-size\": \"10m\", \"max-file\": \"3\" } } docker swarm join --token SWMTKN-1-xxx 192.168.50.114:2377 docker node update --availability drain node docker node update --availability Active node curl --silent --remote-name --location https://github.com/ceph/ceph/raw/octopus/src/cephadm/cephadm chmod +x cephadm sudo ./cephadm add-repo --release octopus sudo ./cephadm install cephadm install ceph-common mkdir -p /etc/ceph sudo ./cephadm bootstrap --mon-ip 192.168.65.19 URL: https://debian.local:8443/ User: admin Password: 12345678 You can access the Ceph CLI with: sudo ./cephadm shell --fsid 8ad7deb6-265a-11ef-9e81-c2d0c41fc7e0 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring Please consider enabling telemetry to help improve Ceph: ceph telemetry on For more information see: https://docs.ceph.com/docs/master/mgr/telemetry/ alias ceph='./cephadm shell -- ceph' sudo ceph -v ceph version 16.2.11 (3cf40e2dca667f68c6ce3ff5cd94f01e711af894) pacific (stable) sudo ceph orch host label add debian mon Added label mon to host debian sudo ceph orch apply mon debian Scheduled mon update... sudo ceph orch host ls ssh-copy-id -f -i /etc/ceph/ceph.pub root@ceph-2 ssh-copy-id -f -i /etc/ceph/ceph.pub osd1 ssh-copy-id -f -i /etc/ceph/ceph.pub osd2 ssh-copy-id -f -i /etc/ceph/ceph.pub osd3 sudo ceph orch host add osd1 sudo ceph orch host add osd2 sudo ceph orch host add osd3 sudo ceph orch apply osd --all-available-devices sudo ceph status sudo find / -iname 'cephadm*' 2>/dev/null systemctl --user start docker-desktop sudo groupadd docker sudo usermod -aG docker $USER groups ${USER} curl -fsSL test.docker.com -o get-docker.sh && sh get-docker.sh sudo apt-get install libffi-dev libssl-dev sudo apt install python3-dev sudo apt-get install -y python3 python3-pip pip install \"cython<3.0.0\" wheel pip install \"pyyaml==5.4.1\" --no-build-isolation pip install docker-compose sudo systemctl enable docker sudo apt-get purge docker-ce sudo apt-get purge docker-ce-cli sudo rm -rf /var/lib/docker rootless sudo apt install -y dbus-user-session sudo apt install -y fuse-overlayfs sudo apt install -y fuse-overlayfs sudo systemctl disable --now docker.service docker.socket sudo rm -rf /var/lib/docker sudo apt-get purge docker-ce docker-ce-cli containerd.io sudo apt-get purge docker-ce docker-ce-cli containerd.io apt-get install -y uidmap curl -fsSL https://get.docker.com/rootless | sh nano .zshrc export PATH=/home/furyhawk/bin:$PATH export DOCKER_HOST=unix:///run/user/1000/docker.sock systemctl --user enable docker sudo loginctl enable-linger $(whoami) docker context use rootless sudo setcap cap_net_bind_service=ep $(which rootlesskit) systemctl --user restart docker #sudo apt-get install -y docker-ce-rootless-extras dockerd-rootless-setuptool.sh uninstall cd ~/bin rm -f containerd containerd-shim containerd-shim-runc-v2 ctr docker docker-init docker-proxy dockerd dockerd-rootless-setuptool.sh dockerd-rootless.sh rootlesskit rootlesskit-docker-proxy runc vpnkit netstat -ltup The Compose Specification {:.no_toc} ToC {:toc} Status of this document This document specifies the Compose file format used to define multi-containers applications. Distribution of this document is unlimited. Requirements and optional attributes The Compose specification includes properties designed to target a local OCI container runtime, exposing Linux kernel specific configuration options, but also some Windows container specific properties. It is also designed for cloud platform features related to resource placement on a cluster, replicated application distribution, and scalability. We acknowledge that no Compose implementation is expected to support all attributes, and that support for some properties is platform dependent and can only be confirmed at runtime. The definition of a versioned schema to control the supported properties in a Compose file, established by the docker-compose tool where the Compose file format was designed, doesn't offer any guarantee to the end-user that attributes will be actually implemented. The specification defines the expected configuration syntax and behavior. Unless noted, supporting any of these is optional. A Compose implementation to parse a Compose file using unsupported attributes should warn users. We recommend the following implementors to support those running modes: Default: warn the user about unsupported attributes, but ignore them Strict: warn the user about unsupported attributes and reject the Compose file Loose: ignore unsupported attributes AND unknown attributes (that were not defined by the spec by the time implementation was created) From this point onwards, references made to 'Compose' can be interpreted as 'a Compose implementation'. The Compose application model The Compose Specification lets you define a platform-agnostic container based application. Such an application is designed as a set of containers which have to both run together with adequate shared resources and communication channels. Computing components of an application are defined as services . A service is an abstract concept implemented on platforms by running the same container image, and configuration, one or more times. Services communicate with each other through networks . In the Compose Specification, a network is a platform capability abstraction to establish an IP route between containers within services connected together. Low-level, platform-specific networking options are grouped into the Network definition and may be partially implemented on some platforms. Services store and share persistent data into volumes . The Specification describes such a persistent data as a high-level filesystem mount with global options. Actual platform-specific implementation details are grouped into the volumes definition and may be partially implemented on some platforms. Some services require configuration data that is dependent on the runtime or platform. For this, the Specification defines a dedicated configs concept. From a service container point of view, configs are comparable to volumes, in that they are files mounted into the container. But the actual definition involves distinct platform resources and services, which are abstracted by this type. A secret is a specific flavor of configuration data for sensitive data that should not be exposed without security considerations. Secrets are made available to services as files mounted into their containers, but the platform-specific resources to provide sensitive data are specific enough to deserve a distinct concept and definition within the Compose specification. Note With volumes, configs and secrets you can have a simple declaration at the top-level and then add more platform-specific information at the service level. A project is an individual deployment of an application specification on a platform. A project's name, set with the top-level name attribute, is used to group resources together and isolate them from other applications or other installation of the same Compose specified application with distinct parameters. If you are creating resources on a platform, you must prefix resource names by project and set the label com.docker.compose.project . Compose offers a way for users to set a custom project name and override this name, so that the same compose.yaml file can be deployed twice on the same infrastructure, without changes, by just passing a distinct name. Project names must contain only lowercase letters, decimal digits, dashes, and underscores, and must begin with a lowercase letter or decimal digit. Illustrative example The following example illustrates the Compose Specification concepts outlined above. The example is non-normative. Consider an application split into a frontend web application and a backend service. The frontend is configured at runtime with an HTTP configuration file managed by infrastructure, providing an external domain name, and an HTTPS server certificate injected by the platform's secured secret store. The backend stores data in a persistent volume. Both services communicate with each other on an isolated back-tier network, while the frontend is also connected to a front-tier network and exposes port 443 for external usage. %%{ init: { 'flowchart': { 'curve': 'linear' } } }%% flowchart LR subgraph A[INFRASTRUCTURE] direction TB subgraph TOP[\" \"] subgraph B1[Frontend Service] fs[\"`**webapp**`\"] end style B1 fill:#ccd6e8, stroke-width:0px subgraph B2[Backend Service] bs[\"`**database**`\"] end style B2 fill:#ccd6e8, stroke-width:0px end style TOP fill:transparent, stroke-width:2px, stroke:#62affb, stroke-dasharray: 5 5 key[ro= read only\\nr+w = read write] style key fill:transparent, stroke-width:0px,text-align: left, size: 94px direction TB id2(Server\\nCertificate) id1(HTTP\\nConfiguration) id1 & id2 -.-|ro| B1 style id1 stroke:#000,stroke-width:1px,stroke-dasharray: 10 style id2 stroke:#000,stroke-width:1px,stroke-dasharray: 10 B2 ==r+w==> id3[(Persistent\\nVolume)] end style A fill:#eeeeee, stroke-width:0px direction LR id4[External\\nUser] ---id5(((443)))--->|Frontend\\nNetwork| B1 style id4 stroke:#000,stroke-width:2px B1 --Backend\\nNetwork--> B2 The example application is composed of the following parts: 2 services, backed by Docker images: webapp and database 1 secret (HTTPS certificate), injected into the frontend 1 configuration (HTTP), injected into the frontend 1 persistent volume, attached to the backend 2 networks services: frontend: image: example/webapp ports: - \"443:8043\" networks: - front-tier - back-tier configs: - httpd-config secrets: - server-certificate backend: image: example/database volumes: - db-data:/etc/data networks: - back-tier volumes: db-data: driver: flocker driver_opts: size: \"10GiB\" configs: httpd-config: external: true secrets: server-certificate: external: true networks: ## The presence of these objects is sufficient to define them front-tier: {} back-tier: {} This example illustrates the distinction between volumes, configs and secrets. While all of them are all exposed to service containers as mounted files or directories, only a volume can be configured for read+write access. Secrets and configs are read-only. The volume configuration allows you to select a volume driver and pass driver options to tweak volume management according to the actual infrastructure. Configs and secrets rely on platform services, and are declared external as they are not managed as part of the application lifecycle. Compose uses a platform-specific lookup mechanism to retrieve runtime values. Compose file The Compose file is a YAML file defining: - Version (Optional) - Services (Required) - Networks - Volumes - Configs - Secrets The default path for a Compose file is compose.yaml (preferred) or compose.yml that is placed in the working directory. Compose also supports docker-compose.yaml and docker-compose.yml for backwards compatibility of earlier versions. If both files exist, Compose prefers the canonical compose.yaml . You can use fragments and extensions to keep your Compose file efficient and easy to maintain. Multiple Compose files can be merged together to define the application model. The combination of YAML files are implemented by appending or overriding YAML elements based on the Compose file order you set. Simple attributes and maps get overridden by the highest order Compose file, lists get merged by appending. Relative paths are resolved based on the first Compose file's parent folder, whenever complimentary files being merged are hosted in other folders. As some Compose file elements can both be expressed as single strings or complex objects, merges apply to the expanded form. If you want to reuse other Compose files, or factor out parts of you application model into separate Compose files, you can also use include . This is useful if your Compose application is dependent on another application which is managed by a different team, or needs to be shared with others. Version and name top-level elements Version top-level element The top-level version property is defined by the Compose Specification for backward compatibility. It is only informative. Compose doesn't use version to select an exact schema to validate the Compose file, but prefers the most recent schema when it's implemented. Compose validates whether it can fully parse the Compose file. If some fields are unknown, typically because the Compose file was written with fields defined by a newer version of the Specification, you'll receive a warning message. Compose offers options to ignore unknown fields (as defined by \"loose\" mode). Name top-level element The top-level name property is defined by the Specification as the project name to be used if you don't set one explicitly. Compose offers a way for you to override this name, and sets a default project name to be used if the top-level name element is not set. Whenever a project name is defined by top-level name or by some custom mechanism, it is exposed for interpolation and environment variable resolution as COMPOSE_PROJECT_NAME services: foo: image: busybox environment: - COMPOSE_PROJECT_NAME command: echo \"I'm running ${COMPOSE_PROJECT_NAME}\" Services top-level element A service is an abstract definition of a computing resource within an application which can be scaled or replaced independently from other components. Services are backed by a set of containers, run by the platform according to replication requirements and placement constraints. As services are backed by containers, they are defined by a Docker image and set of runtime arguments. All containers within a service are identically created with these arguments. A Compose file must declare a services top-level element as a map whose keys are string representations of service names, and whose values are service definitions. A service definition contains the configuration that is applied to each service container. Each service may also include a build section, which defines how to create the Docker image for the service. Compose supports building docker images using this service definition. If not used, the build section is ignored and the Compose file is still considered valid. Build support is an optional aspect of the Compose Specification, and is described in detail in the Compose Build Specification documentation. Each service defines runtime constraints and requirements to run its containers. The deploy section groups these constraints and allows the platform to adjust the deployment strategy to best match containers' needs with available resources. Deploy support is an optional aspect of the Compose Specification, and is described in detail in the Compose Deploy Specification documentation. If not implemented the deploy section is ignored and the Compose file is still considered valid. attach When attach is defined and set to false Compose does not collect service logs, until you explicitly request it to. The default service configuration is attach: true . build build specifies the build configuration for creating a container image from source, as defined in the Compose Build Specification . blkio_config blkio_config defines a set of configuration options to set block IO limits for a service. services: foo: image: busybox blkio_config: weight: 300 weight_device: - path: /dev/sda weight: 400 device_read_bps: - path: /dev/sdb rate: '12mb' device_read_iops: - path: /dev/sdb rate: 120 device_write_bps: - path: /dev/sdb rate: '1024k' device_write_iops: - path: /dev/sdb rate: 30 device_read_bps, device_write_bps Set a limit in bytes per second for read / write operations on a given device. Each item in the list must have two keys: path : Defines the symbolic path to the affected device. rate : Either as an integer value representing the number of bytes or as a string expressing a byte value. device_read_iops, device_write_iops Set a limit in operations per second for read / write operations on a given device. Each item in the list must have two keys: path : Defines the symbolic path to the affected device. rate : As an integer value representing the permitted number of operations per second. weight Modify the proportion of bandwidth allocated to a service relative to other services. Takes an integer value between 10 and 1000, with 500 being the default. weight_device Fine-tune bandwidth allocation by device. Each item in the list must have two keys: path : Defines the symbolic path to the affected device. weight : An integer value between 10 and 1000. cpu_count cpu_count defines the number of usable CPUs for service container. cpu_percent cpu_percent defines the usable percentage of the available CPUs. cpu_shares cpu_shares defines, as integer value, a service container's relative CPU weight versus other containers. cpu_period cpu_period configures CPU CFS (Completely Fair Scheduler) period when a platform is based on Linux kernel. cpu_quota cpu_quota configures CPU CFS (Completely Fair Scheduler) quota when a platform is based on Linux kernel. cpu_rt_runtime cpu_rt_runtime configures CPU allocation parameters for platforms with support for realtime scheduler. It can be either an integer value using microseconds as unit or a duration . cpu_rt_runtime: '400ms' cpu_rt_runtime: 95000` cpu_rt_period cpu_rt_period configures CPU allocation parameters for platforms with support for realtime scheduler. It can be either an integer value using microseconds as unit or a duration . cpu_rt_period: '1400us' cpu_rt_period: 11000` cpus DEPRECATED: use deploy.limits.cpus cpus define the number of (potentially virtual) CPUs to allocate to service containers. This is a fractional number. 0.000 means no limit. cpuset cpuset defines the explicit CPUs in which to allow execution. Can be a range 0-3 or a list 0,1 cap_add cap_add specifies additional container capabilities as strings. cap_add: - ALL cap_drop cap_drop specifies container capabilities to drop as strings. cap_drop: - NET_ADMIN - SYS_ADMIN cgroup cgroup specifies the cgroup namespace to join. When unset, it is the container runtime's decision to select which cgroup namespace to use, if supported. host : Runs the container in the Container runtime cgroup namespace. private : Runs the container in its own private cgroup namespace. cgroup_parent cgroup_parent specifies an optional parent cgroup for the container. cgroup_parent: m-executor-abcd command command overrides the default command declared by the container image, for example by Dockerfile's CMD . command: bundle exec thin -p 3000 The value can also be a list, in a manner similar to Dockerfile : command: [ \"bundle\", \"exec\", \"thin\", \"-p\", \"3000\" ] If the value is null , the default command from the image is used. If the value is [] (empty list) or '' (empty string), the default command declared by the image is ignored, i.e. overridden to be empty. configs Configs allow services to adapt their behaviour without the need to rebuild a Docker image. Services can only access configs when explicitly granted by the configs attribute. Two different syntax variants are supported. Compose reports an error if config doesn't exist on the platform or isn't defined in the configs top-level element in the Compose file. There are two syntaxes defined for configs. To remain compliant to this specification, an implementation must support both syntaxes. Implementations must allow use of both short and long syntaxes within the same document. You can grant a service access to multiple configs, and you can mix long and short syntax. Short syntax The short syntax variant only specifies the config name. This grants the container access to the config and mounts it as files into a service\u2019s container\u2019s filesystem. The location of the mount point within the container defaults to / in Linux containers, and C:\\ in Windows containers. The following example uses the short syntax to grant the redis service access to the my_config and my_other_config configs. The value of my_config is set to the contents of the file ./my_config.txt , and my_other_config is defined as an external resource, which means that it has already been defined in the platform. If the external config does not exist, the deployment fails. services: redis: image: redis:latest configs: - my_config - my_other_config configs: my_config: file: ./my_config.txt my_other_config: external: true Long syntax The long syntax provides more granularity in how the config is created within the service's task containers. source : The name of the config as it exists in the platform. target : The path and name of the file to be mounted in the service's task containers. Defaults to / if not specified. uid and gid : The numeric UID or GID that owns the mounted config file within the service's task containers. Default value when not specified is USER running container. mode : The permissions for the file that is mounted within the service's task containers, in octal notation. Default value is world-readable ( 0444 ). Writable bit must be ignored. The executable bit can be set. The following example sets the name of my_config to redis_config within the container, sets the mode to 0440 (group-readable) and sets the user and group to 103 . The redis service does not have access to the my_other_config config. services: redis: image: redis:latest configs: - source: my_config target: /redis_config uid: \"103\" gid: \"103\" mode: 0440 configs: my_config: external: true my_other_config: external: true container_name container_name is a string that specifies a custom container name, rather than a name generated by default. container_name: my-web-container Compose does not scale a service beyond one container if the Compose file specifies a container_name . Attempting to do so results in an error. container_name follows the regex format of [a-zA-Z0-9][a-zA-Z0-9_.-]+ credential_spec credential_spec configures the credential spec for a managed service account. If you have services that use Windows containers, you can use file: and registry: protocols for credential_spec . Compose also supports additional protocols for custom use-cases. The credential_spec must be in the format file:// or registry:// . credential_spec: file: my-credential-spec.json When using registry: , the credential spec is read from the Windows registry on the daemon's host. A registry value with the given name must be located in: HKLM\\SOFTWARE\\Microsoft\\Windows NT\\CurrentVersion\\Virtualization\\Containers\\CredentialSpecs The following example loads the credential spec from a value named my-credential-spec in the registry: credential_spec: registry: my-credential-spec Example gMSA configuration When configuring a gMSA credential spec for a service, you only need to specify a credential spec with config , as shown in the following example: services: myservice: image: myimage:latest credential_spec: config: my_credential_spec configs: my_credentials_spec: file: ./my-credential-spec.json| depends_on depends_on expresses startup and shutdown dependencies between services. Short syntax The short syntax variant only specifies service names of the dependencies. Service dependencies cause the following behaviors: Compose creates services in dependency order. In the following example, db and redis are created before web . Compose removes services in dependency order. In the following example, web is removed before db and redis . Simple example: services: web: build: . depends_on: - db - redis redis: image: redis db: image: postgres Compose guarantees dependency services have been started before starting a dependent service. Compose waits for dependency services to be \"ready\" before starting a dependent service. Long syntax The long form syntax enables the configuration of additional fields that can't be expressed in the short form. restart : When set to true Compose restarts this service after it updates the dependency service. This applies to an explicit restart controlled by a Compose operation, and excludes automated restart by the container runtime after the container dies. condition : Sets the condition under which dependency is considered satisfied service_started : An equivalent of the short syntax described above service_healthy : Specifies that a dependency is expected to be \"healthy\" (as indicated by healthcheck ) before starting a dependent service. service_completed_successfully : Specifies that a dependency is expected to run to successful completion before starting a dependent service. required : When set to false Compose only warns you when the dependency service isn't started or available. If it's not defined the default value of required is true . Service dependencies cause the following behaviors: Compose creates services in dependency order. In the following example, db and redis are created before web . Compose waits for healthchecks to pass on dependencies marked with service_healthy . In the following example, db is expected to be \"healthy\" before web is created. Compose removes services in dependency order. In the following example, web is removed before db and redis . services: web: build: . depends_on: db: condition: service_healthy restart: true redis: condition: service_started redis: image: redis db: image: postgres Compose guarantees dependency services are started before starting a dependent service. Compose guarantees dependency services marked with service_healthy are \"healthy\" before starting a dependent service. deploy deploy specifies the configuration for the deployment and lifecycle of services, as defined in the Compose Deploy Specification . develop develop specifies the development configuration for maintaining a container in sync with source, as defined in the Development Section . device_cgroup_rules device_cgroup_rules defines a list of device cgroup rules for this container. The format is the same format the Linux kernel specifies in the Control Groups Device Whitelist Controller . device_cgroup_rules: - 'c 1:3 mr' - 'a 7:* rmw' devices devices defines a list of device mappings for created containers in the form of HOST_PATH:CONTAINER_PATH[:CGROUP_PERMISSIONS] . devices: - \"/dev/ttyUSB0:/dev/ttyUSB0\" - \"/dev/sda:/dev/xvda:rwm\" dns dns defines custom DNS servers to set on the container network interface configuration. It can be a single value or a list. dns: 8.8.8.8 dns: - 8.8.8.8 - 9.9.9.9 dns_opt dns_opt list custom DNS options to be passed to the container\u2019s DNS resolver ( /etc/resolv.conf file on Linux). dns_opt: - use-vc - no-tld-query dns_search dns_search defines custom DNS search domains to set on container network interface configuration. It can be a single value or a list. dns_search: example.com dns_search: - dc1.example.com - dc2.example.com domainname domainname declares a custom domain name to use for the service container. It must be a valid RFC 1123 hostname. entrypoint entrypoint declares the default entrypoint for the service container. This overrides the ENTRYPOINT instruction from the service's Dockerfile. If entrypoint is non-null, Compose ignores any default command from the image, for example the CMD instruction in the Dockerfile. See also command to set or override the default command to be executed by the entrypoint process. In its short form, the value can be defined as a string: entrypoint: /code/entrypoint.sh Alternatively, the value can also be a list, in a manner similar to the Dockerfile : entrypoint: - php - -d - zend_extension=/usr/local/lib/php/extensions/no-debug-non-zts-20100525/xdebug.so - -d - memory_limit=-1 - vendor/bin/phpunit If the value is null , the default entrypoint from the image is used. If the value is [] (empty list) or '' (empty string), the default entrypoint declared by the image is ignored, i.e. overridden to be empty. env_file env_file adds environment variables to the container based on the file content. env_file: .env env_file can also be a list. The files in the list are processed from the top down. For the same variable specified in two env files, the value from the last file in the list stands. env_file: - ./a.env - ./b.env List elements can also be declared as a mapping, which then lets you set an additional attribute required . This defaults to true . When required is set to false and the .env file is missing, Compose silently ignores the entry. env_file: - path: ./default.env required: true ## default - path: ./override.env required: false Relative path are resolved from the Compose file's parent folder. As absolute paths prevent the Compose file from being portable, Compose warns you when such a path is used to set env_file . Environment variables declared in the environment section override these values. This holds true even if those values are empty or undefined. Env_file format Each line in an .env file must be in VAR[=[VAL]] format. The following syntax rules apply: Lines beginning with # are processed as comments and ignored. Blank lines are ignored. Unquoted and double-quoted ( \" ) values have Interpolation applied. Each line represents a key-value pair. Values can optionally be quoted. VAR=VAL -> VAL VAR=\"VAL\" -> VAL VAR='VAL' -> VAL Inline comments for unquoted values must be preceded with a space. VAR=VAL ## comment -> VAL VAR=VAL## not a comment -> VAL## not a comment Inline comments for quoted values must follow the closing quote. VAR=\"VAL ## not a comment\" -> VAL ## not a comment VAR=\"VAL\" ## comment -> VAL Single-quoted ( ' ) values are used literally. VAR='$OTHER' -> $OTHER VAR='${OTHER}' -> ${OTHER} Quotes can be escaped with \\ . VAR='Let\\'s go!' -> Let's go! VAR=\"{\\\"hello\\\": \\\"json\\\"}\" -> {\"hello\": \"json\"} Common shell escape sequences including \\n , \\r , \\t , and \\\\ are supported in double-quoted values. VAR=\"some\\tvalue\" -> some value VAR='some\\tvalue' -> some\\tvalue VAR=some\\tvalue -> some\\tvalue VAL may be omitted, in such cases the variable value is an empty string. =VAL may be omitted, in such cases the variable is unset. ## Set Rails/Rack environment RACK_ENV=development VAR=\"quoted\" environment environment defines environment variables set in the container. environment can use either an array or a map. Any boolean values; true, false, yes, no, should be enclosed in quotes to ensure they are not converted to True or False by the YAML parser. Environment variables can be declared by a single key (no value to equals sign). In this case Compose relies on you to resolve the value. If the value is not resolved, the variable is unset and is removed from the service container environment. Map syntax: environment: RACK_ENV: development SHOW: \"true\" USER_INPUT: Array syntax: environment: - RACK_ENV=development - SHOW=true - USER_INPUT When both env_file and environment are set for a service, values set by environment have precedence. expose expose defines the (incoming) port or a range of ports that Compose exposes from the container. These ports must be accessible to linked services and should not be published to the host machine. Only the internal container ports can be specified. Syntax is /[] or /[] for a port range. When not explicitly set, tcp protocol is used. expose: - \"3000\" - \"8000\" - \"8080-8085/tcp Note If the Dockerfile for the image already exposes ports, it is visible to other containers on the network even if expose is not set in your Compose file. extends extends lets you share common configurations among different files, or even different projects entirely. With extends you can define a common set of service options in one place and refer to it from anywhere. You can refer to another Compose file and select a service you want to also use in your own application, with the ability to override some attributes for your own needs. You can use extends on any service together with other configuration keys. The extends value must be a mapping defined with a required service and an optional file key. extends: file: common.yml service: webapp service : Defines the name of the service being referenced as a base, for example web or database . file : The location of a Compose configuration file defining that service. Restrictions The following restrictions apply to the service being referenced: Services that have dependencies on other services cannot be used as a base. Therefore, any key that introduces a dependency on another service is incompatible with extends . The non-exhaustive list of such keys is: links , volumes_from , container mode (in ipc , pid , network_mode and net ), service mode (in ipc , pid and network_mode ), depends_on . Services cannot have circular references with extends . Compose returns an error in all of these cases. Finding referenced service file value can be: Not present. This indicates that another service within the same Compose file is being referenced. File path, which can be either: Relative path. This path is considered as relative to the location of the main Compose file. Absolute path. A service denoted by service must be present in the identified referenced Compose file. Compose returns an error if: The service denoted by service is not found. The Compose file denoted by file is not found. Merging service definitions Two service definitions, the main one in the current Compose file and the referenced one specified by extends , are merged in the following way: Mappings: Keys in mappings of the main service definition override keys in mappings of the referenced service definition. Keys that aren't overridden are included as is. Sequences: Items are combined together into a new sequence. The order of elements is preserved with the referenced items coming first and main items after. Scalars: Keys in the main service definition take precedence over keys in the referenced one. Mappings The following keys should be treated as mappings: annotations , build.args , build.labels , build.extra_hosts , deploy.labels , deploy.update_config , deploy.rollback_config , deploy.restart_policy , deploy.resources.limits , environment , healthcheck , labels , logging.options , sysctls , storage_opt , extra_hosts , ulimits . One exception that applies to healthcheck is that the main mapping cannot specify disable: true unless the referenced mapping also specifies disable: true . Compose returns an error in this case. For example, the input below: services: common: image: busybox environment: TZ: utc PORT: 80 cli: extends: service: common environment: PORT: 8080 Produces the following configuration for the cli service. The same output is produced if array syntax is used. environment: PORT: 8080 TZ: utc image: busybox Items under blkio_config.device_read_bps , blkio_config.device_read_iops , blkio_config.device_write_bps , blkio_config.device_write_iops , devices and volumes are also treated as mappings where key is the target path inside the container. For example, the input below: services: common: image: busybox volumes: - common-volume:/var/lib/backup/data:rw cli: extends: service: common volumes: - cli-volume:/var/lib/backup/data:ro Produces the following configuration for the cli service. Note that the mounted path now points to the new volume name and ro flag was applied. image: busybox volumes: - cli-volume:/var/lib/backup/data:ro If the referenced service definition contains extends mapping, the items under it are simply copied into the new merged definition. The merging process is then kicked off again until no extends keys are remaining. For example, the input below: services: base: image: busybox user: root common: image: busybox extends: service: base cli: extends: service: common Produces the following configuration for the cli service. Here, cli services gets user key from common service, which in turn gets this key from base service. image: busybox user: root Sequences The following keys should be treated as sequences: cap_add , cap_drop , configs , deploy.placement.constraints , deploy.placement.preferences , deploy.reservations.generic_resources , device_cgroup_rules , expose , external_links , ports , secrets , security_opt . Any duplicates resulting from the merge are removed so that the sequence only contains unique elements. For example, the input below: services: common: image: busybox security_opt: - label:role:ROLE cli: extends: service: common security_opt: - label:user:USER Produces the following configuration for the cli service. image: busybox security_opt: - label:role:ROLE - label:user:USER In case list syntax is used, the following keys should also be treated as sequences: dns , dns_search , env_file , tmpfs . Unlike sequence fields mentioned above, duplicates resulting from the merge are not removed. Scalars Any other allowed keys in the service definition should be treated as scalars. annotations annotations defines annotations for the container. annotations can use either an array or a map. annotations: com.example.foo: bar annotations: - com.example.foo=bar external_links external_links link service containers to services managed outside of your Compose application. external_links define the name of an existing service to retrieve using the platform lookup mechanism. An alias of the form SERVICE:ALIAS can be specified. external_links: - redis - database:mysql - database:postgresql extra_hosts extra_hosts adds hostname mappings to the container network interface configuration ( /etc/hosts for Linux). Short syntax Short syntax uses plain strings in a list. Values must set hostname and IP address for additional hosts in the form of HOSTNAME=IP . extra_hosts: - \"somehost=162.242.195.82\" - \"otherhost=50.31.209.229\" - \"myhostv6=::1\" IPv6 addresses can be enclosed in square brackets, for example: extra_hosts: - \"myhostv6=[::1]\" The separator = is preferred, but : can also be used. For example: extra_hosts: - \"somehost:162.242.195.82\" - \"myhostv6:::1\" Long syntax Alternatively, extra_hosts can be set as a mapping between hostname(s) and IP(s) extra_hosts: somehost: \"162.242.195.82\" otherhost: \"50.31.209.229\" myhostv6: \"::1\" Compose creates a matching entry with the IP address and hostname in the container's network configuration, which means for Linux /etc/hosts get extra lines: 162.242.195.82 somehost 50.31.209.229 otherhost ::1 myhostv6 group_add group_add specifies additional groups, by name or number, which the user inside the container must be a member of. An example of where this is useful is when multiple containers (running as different users) need to all read or write the same file on a shared volume. That file can be owned by a group shared by all the containers, and specified in group_add . services: myservice: image: alpine group_add: - mail Running id inside the created container must show that the user belongs to the mail group, which would not have been the case if group_add were not declared. healthcheck healthcheck declares a check that's run to determine whether or not the service containers are \"healthy\". It works in the same way, and has the same default values, as the HEALTHCHECK Dockerfile instruction set by the service's Docker image. Your Compose file can override the values set in the Dockerfile. healthcheck: test: [\"CMD\", \"curl\", \"-f\", \"http://localhost\"] interval: 1m30s timeout: 10s retries: 3 start_period: 40s start_interval: 5s interval , timeout , start_period , and start_interval are specified as durations . test defines the command Compose runs to check container health. It can be either a string or a list. If it's a list, the first item must be either NONE , CMD or CMD-SHELL . If it's a string, it's equivalent to specifying CMD-SHELL followed by that string. ## Hit the local web app test: [\"CMD\", \"curl\", \"-f\", \"http://localhost\"] Using CMD-SHELL runs the command configured as a string using the container's default shell ( /bin/sh for Linux). Both forms below are equivalent: test: [\"CMD-SHELL\", \"curl -f http://localhost || exit 1\"] test: curl -f https://localhost || exit 1 NONE disables the healthcheck, and is mostly useful to disable the Healthcheck Dockerfile instruction set by the service's Docker image. Alternatively, the healthcheck set by the image can be disabled by setting disable: true : healthcheck: disable: true hostname hostname declares a custom host name to use for the service container. It must be a valid RFC 1123 hostname. image image specifies the image to start the container from. image must follow the Open Container Specification addressable image format , as [/][/][:|@] . image: redis image: redis:5 image: redis@sha256:0ed5d5928d4737458944eb604cc8509e245c3e19d02ad83935398bc4b991aac7 image: library/redis image: docker.io/library/redis image: my_private.registry:5000/redis If the image does not exist on the platform, Compose attempts to pull it based on the pull_policy . If you are also using the Compose Build Specification , there are alternative options for controlling the precedence of pull over building the image from source, however pulling the image is the default behavior. image may be omitted from a Compose file as long as a build section is declared. If you are not using the Compose Build Specification, Compose won't work if image is missing from the Compose file. init init runs an init process (PID 1) inside the container that forwards signals and reaps processes. Set this option to true to enable this feature for the service. services: web: image: alpine:latest init: true The init binary that is used is platform specific. ipc ipc configures the IPC isolation mode set by the service container. Available values are platform specific, but Compose defines specific values which must be implemented as described if supported: shareable : Gives the container its own private IPC namespace, with a possibility to share it with other containers. service:{name} : Makes the container join another container's ( shareable ) IPC namespace. ipc: \"shareable\" ipc: \"service:[service name]\" uts uts configures the UTS namespace mode set for the service container. When unspecified it is the runtime's decision to assign a UTS namespace, if supported. Available values are: 'host' : Results in the container using the same UTS namespace as the host. uts: \"host\" isolation isolation specifies a container\u2019s isolation technology. Supported values are platform specific. labels labels add metadata to containers. You can use either an array or a map. It's recommended that you use reverse-DNS notation to prevent your labels from conflicting with those used by other software. labels: com.example.description: \"Accounting webapp\" com.example.department: \"Finance\" com.example.label-with-empty-value: \"\" labels: - \"com.example.description=Accounting webapp\" - \"com.example.department=Finance\" - \"com.example.label-with-empty-value\" Compose creates containers with canonical labels: com.docker.compose.project set on all resources created by Compose to the user project name com.docker.compose.service set on service containers with service name as defined in the Compose file The com.docker.compose label prefix is reserved. Specifying labels with this prefix in the Compose file results in a runtime error. links Note Availability of the links attribute is implementation specific. links defines a network link to containers in another service. Either specify both the service name and a link alias ( SERVICE:ALIAS ), or just the service name. web: links: - db - db:database - redis Containers for the linked service are reachable at a hostname identical to the alias, or the service name if no alias is specified. Links are not required to enable services to communicate. When no specific network configuration is set, any service is able to reach any other service at that service\u2019s name on the default network. If services do declare networks they are attached to, links does not override the network configuration and services not attached to a shared network are not be able to communicate. Compose doesn't warn you about a configuration mismatch. Links also express implicit dependency between services in the same way as depends_on , so they determine the order of service startup. logging logging defines the logging configuration for the service. logging: driver: syslog options: syslog-address: \"tcp://192.168.0.42:123\" The driver name specifies a logging driver for the service's containers. The default and available values are platform specific. Driver specific options can be set with options as key-value pairs. network_mode network_mode sets a service container's network mode. Available values are platform specific, but Compose defines specific values which must be implemented as described if supported: none : Turns off all container networking. host : Gives the container raw access to the host's network interface. service:{name} : Gives the containers access to the specified service only. network_mode: \"host\" network_mode: \"none\" network_mode: \"service:[service name]\" When set, the networks attribute is not allowed and Compose rejects any Compose file containing both attributes. networks networks defines the networks that service containers are attached to, referencing entries under the top-level networks key . services: some-service: networks: - some-network - other-network aliases aliases declares alternative hostnames for the service on the network. Other containers on the same network can use either the service name or an alias to connect to one of the service's containers. Since aliases are network-scoped, the same service can have different aliases on different networks. Note A network-wide alias can be shared by multiple containers, and even by multiple services. If it is, then exactly which container the name resolves to is not guaranteed. services: some-service: networks: some-network: aliases: - alias1 - alias3 other-network: aliases: - alias2 In the following example, service frontend is able to reach the backend service at the hostname backend or database on the back-tier network. The service monitoring is able to reach same backend service at backend or mysql on the admin network. services: frontend: image: example/webapp networks: - front-tier - back-tier monitoring: image: example/monitoring networks: - admin backend: image: example/backend networks: back-tier: aliases: - database admin: aliases: - mysql networks: front-tier: back-tier: admin: ipv4_address, ipv6_address Specify a static IP address for a service container when joining the network. The corresponding network configuration in the top-level networks section must have an ipam attribute with subnet configurations covering each static address. services: frontend: image: example/webapp networks: front-tier: ipv4_address: 172.16.238.10 ipv6_address: 2001:3984:3989::10 networks: front-tier: ipam: driver: default config: - subnet: \"172.16.238.0/24\" - subnet: \"2001:3984:3989::/64\" link_local_ips link_local_ips specifies a list of link-local IPs. Link-local IPs are special IPs which belong to a well known subnet and are purely managed by the operator, usually dependent on the architecture where they are deployed. Implementation is platform specific. Example: services: app: image: busybox command: top networks: app_net: link_local_ips: - 57.123.22.11 - 57.123.22.13 networks: app_net: driver: bridge mac_address mac_address sets the MAC address used by the service container when connecting to this particular network. priority priority indicates in which order Compose connects the service\u2019s containers to its networks. If unspecified, the default value is 0. In the following example, the app service connects to app_net_1 first as it has the highest priority. It then connects to app_net_3 , then app_net_2 , which uses the default priority value of 0. services: app: image: busybox command: top networks: app_net_1: priority: 1000 app_net_2: app_net_3: priority: 100 networks: app_net_1: app_net_2: app_net_3: mac_address mac_address sets a MAC address for the service container. Note Container runtimes might reject this value (ie. Docker Engine >= v25.0). In that case, you should use networks.mac_address instead. mem_limit DEPRECATED: use deploy.limits.memory mem_reservation DEPRECATED: use deploy.reservations.memory mem_swappiness mem_swappiness defines as a percentage, a value between 0 and 100, for the host kernel to swap out anonymous memory pages used by a container. 0 : Turns off anonymous page swapping. 100 : Sets all anonymous pages as swappable. The default value is platform specific. memswap_limit memswap_limit defines the amount of memory the container is allowed to swap to disk. This is a modifier attribute that only has meaning if memory is also set. Using swap lets the container write excess memory requirements to disk when the container has exhausted all the memory that is available to it. There is a performance penalty for applications that swap memory to disk often. If memswap_limit is set to a positive integer, then both memory and memswap_limit must be set. memswap_limit represents the total amount of memory and swap that can be used, and memory controls the amount used by non-swap memory. So if memory =\"300m\" and memswap_limit =\"1g\", the container can use 300m of memory and 700m (1g - 300m) swap. If memswap_limit is set to 0, the setting is ignored, and the value is treated as unset. If memswap_limit is set to the same value as memory , and memory is set to a positive integer, the container does not have access to swap. If memswap_limit is unset, and memory is set, the container can use as much swap as the memory setting, if the host container has swap memory configured. For instance, if memory =\"300m\" and memswap_limit is not set, the container can use 600m in total of memory and swap. If memswap_limit is explicitly set to -1, the container is allowed to use unlimited swap, up to the amount available on the host system. oom_kill_disable If oom_kill_disable is set, Compose configures the platform so it won't kill the container in case of memory starvation. oom_score_adj oom_score_adj tunes the preference for containers to be killed by platform in case of memory starvation. Value must be within [-1000,1000] range. pid pid sets the PID mode for container created by Compose. Supported values are platform specific. pids_limit DEPRECATED: use deploy.resources.limits.pids pids_limit tunes a container\u2019s PIDs limit. Set to -1 for unlimited PIDs. pids_limit: 10 platform platform defines the target platform the containers for the service run on. It uses the os[/arch[/variant]] syntax. The values of os , arch , and variant must conform to the convention used by the OCI Image Spec . Compose uses this attribute to determine which version of the image is pulled and/or on which platform the service\u2019s build is performed. platform: darwin platform: windows/amd64 platform: linux/arm64/v8 ports Exposes container ports. Note Port mapping must not be used with network_mode: host otherwise a runtime error occurs. Short syntax The short syntax is a colon-separated string to set the host IP, host port, and container port in the form: [HOST:]CONTAINER[/PROTOCOL] where: HOST is [IP:](port | range) CONTAINER is port | range PROTOCOL to restrict port to specified protocol. tcp and udp values are defined by the Specification, Compose offers support for platform-specific protocol names. If host IP is not set, it binds to all network interfaces. Ports can be either a single value or a range. Host and container must use equivalent ranges. Either specify both ports ( HOST:CONTAINER ), or just the container port. In the latter case, the container runtime automatically allocates any unassigned port of the host. HOST:CONTAINER should always be specified as a (quoted) string, to avoid conflicts with yaml base-60 float . Examples: ports: - \"3000\" - \"3000-3005\" - \"8000:8000\" - \"9090-9091:8080-8081\" - \"49100:22\" - \"8000-9000:80\" - \"127.0.0.1:8001:8001\" - \"127.0.0.1:5000-5010:5000-5010\" - \"6060:6060/udp\" Note If Host IP mapping is not supported by a container engine, Compose rejects the Compose file and ignores the specified host IP. Long syntax The long form syntax allows the configuration of additional fields that can't be expressed in the short form. target : The container port published : The publicly exposed port. It is defined as a string and can be set as a range using syntax start-end . It means the actual port is assigned a remaining available port, within the set range. host_ip : The Host IP mapping, unspecified means all network interfaces ( 0.0.0.0 ). protocol : The port protocol ( tcp or udp ). Defaults to tcp . mode : host : For publishing a host port on each node, or ingress for a port to be load balanced. Defaults to ingress . name : A human-readable name for the port, used to document it's usage within the service ports: - name: http target: 80 host_ip: 127.0.0.1 published: \"8080\" protocol: tcp mode: host - name: https target: 443 host_ip: 127.0.0.1 published: \"8083-9000\" protocol: tcp mode: host privileged privileged configures the service container to run with elevated privileges. Support and actual impacts are platform specific. profiles profiles defines a list of named profiles for the service to be enabled under. If unassigned, the service is always started but if assigned, it is only started if the profile is activated. If present, profiles follow the regex format of [a-zA-Z0-9][a-zA-Z0-9_.-]+ . services: frontend: image: frontend profiles: [\"frontend\"] phpmyadmin: image: phpmyadmin depends_on: - db profiles: - debug pull_policy pull_policy defines the decisions Compose makes when it starts to pull images. Possible values are: always : Compose always pulls the image from the registry. never : Compose doesn't pull the image from a registry and relies on the platform cached image. If there is no cached image, a failure is reported. missing : Compose pulls the image only if it's not available in the platform cache. This is the default option if you are not also using the Compose Build Specification . if_not_present is considered an alias for this value for backward compatibility. build : Compose builds the image. Compose rebuilds the image if it's already present. read_only read_only configures the service container to be created with a read-only filesystem. restart restart defines the policy that the platform applies on container termination. no : The default restart policy. It does not restart the container under any circumstances. always : The policy always restarts the container until its removal. on-failure : The policy restarts the container if the exit code indicates an error. unless-stopped : The policy restarts the container irrespective of the exit code but stops restarting when the service is stopped or removed. restart: \"no\" restart: always restart: on-failure restart: unless-stopped runtime runtime specifies which runtime to use for the service\u2019s containers. The value of runtime is specific to the implementation. For example, runtime can be the name of an implementation of OCI Runtime Spec , such as \"runc\". web: image: busybox:latest command: true runtime: runc scale scale specifies the default number of containers to deploy for this service. When both are set, scale must be consistent with the replicas attribute in the Deploy Specification . secrets secrets grants access to sensitive data defined by secrets on a per-service basis. Two different syntax variants are supported; the short syntax and the long syntax. Compose reports an error if the secret doesn't exist on the platform or isn't defined in the secrets section of the Compose file. Services can be granted access to multiple secrets. Long and short syntax for secrets may be used in the same Compose file. Defining a secret in the top-level secrets must not imply granting any service access to it. Such grant must be explicit within service specification as secrets service element. Short syntax The short syntax variant only specifies the secret name. This grants the container access to the secret and mounts it as read-only to /run/secrets/ within the container. The source name and destination mountpoint are both set to the secret name. The following example uses the short syntax to grant the frontend service access to the server-certificate secret. The value of server-certificate is set to the contents of the file ./server.cert . services: frontend: image: example/webapp secrets: - server-certificate secrets: server-certificate: file: ./server.cert Long syntax The long syntax provides more granularity in how the secret is created within the service's containers. source : The name of the secret as it exists on the platform. target : The name of the file to be mounted in /run/secrets/ in the service's task container, or absolute path of the file if an alternate location is required. Defaults to source if not specified. uid and gid : The numeric UID or GID that owns the file within /run/secrets/ in the service's task containers. Default value is USER running container. mode : The permissions for the file to be mounted in /run/secrets/ in the service's task containers, in octal notation. The default value is world-readable permissions (mode 0444 ). The writable bit must be ignored if set. The executable bit may be set. Note that the uid , gid , and mode attributes are implementation specific. The following example sets the name of the server-certificate secret file to server.crt within the container, sets the mode to 0440 (group-readable), and sets the user and group to 103 . The value of server-certificate secret is provided by the platform through a lookup and the secret's lifecycle is not directly managed by Compose. services: frontend: image: example/webapp secrets: - source: server-certificate target: server.cert uid: \"103\" gid: \"103\" mode: 0440 secrets: server-certificate: external: true security_opt security_opt overrides the default labeling scheme for each container. security_opt: - label:user:USER - label:role:ROLE For further default labeling schemes you can override, see Security configuration . shm_size shm_size configures the size of the shared memory ( /dev/shm partition on Linux) allowed by the service container. It's specified as a byte value . stdin_open stdin_open configures a service containers to run with an allocated stdin. stop_grace_period stop_grace_period specifies how long Compose must wait when attempting to stop a container if it doesn't handle SIGTERM (or whichever stop signal has been specified with stop_signal ), before sending SIGKILL. It's specified as a duration . stop_grace_period: 1s stop_grace_period: 1m30s Default value is 10 seconds for the container to exit before sending SIGKILL. stop_signal stop_signal defines the signal that Compose uses to stop the service containers. If unset containers are stopped by Compose by sending SIGTERM . stop_signal: SIGUSR1 storage_opt storage_opt defines storage driver options for a service. storage_opt: size: '1G' sysctls sysctls defines kernel parameters to set in the container. sysctls can use either an array or a map. sysctls: net.core.somaxconn: 1024 net.ipv4.tcp_syncookies: 0 sysctls: - net.core.somaxconn=1024 - net.ipv4.tcp_syncookies=0 You can only use sysctls that are namespaced in the kernel. Docker does not support changing sysctls inside a container that also modify the host system. For an overview of supported sysctls, refer to configure namespaced kernel parameters (sysctls) at runtime . tmpfs tmpfs mounts a temporary file system inside the container. It can be a single value or a list. tmpfs: /run tmpfs: - /run - /tmp tty tty configures service container to run with a TTY. ulimits ulimits overrides the default ulimits for a container. It's specified either as an integer for a single limit or as mapping for soft/hard limits. ulimits: nproc: 65535 nofile: soft: 20000 hard: 40000 user user overrides the user used to run the container process. The default is set by the image (i.e. Dockerfile USER ). If it's not set, then root . userns_mode userns_mode sets the user namespace for the service. Supported values are platform specific and may depend on platform configuration. userns_mode: \"host\" volumes volumes define mount host paths or named volumes that are accessible by service containers. You can use volumes to define multiple types of mounts; volume , bind , tmpfs , or npipe . If the mount is a host path and is only used by a single service, it can be declared as part of the service definition. To reuse a volume across multiple services, a named volume must be declared in the top-level volumes key . The following example shows a named volume ( db-data ) being used by the backend service, and a bind mount defined for a single service. services: backend: image: example/backend volumes: - type: volume source: db-data target: /data volume: nocopy: true - type: bind source: /var/run/postgres/postgres.sock target: /var/run/postgres/postgres.sock volumes: db-data: Short syntax The short syntax uses a single string with colon-separated values to specify a volume mount ( VOLUME:CONTAINER_PATH ), or an access mode ( VOLUME:CONTAINER_PATH:ACCESS_MODE ). VOLUME : Can be either a host path on the platform hosting containers (bind mount) or a volume name. CONTAINER_PATH : The path in the container where the volume is mounted. ACCESS_MODE : A comma-separated , list of options: rw : Read and write access. This is the default if none is specified. ro : Read-only access. z : SELinux option indicating that the bind mount host content is shared among multiple containers. Z : SELinux option indicating that the bind mount host content is private and unshared for other containers. Note The SELinux re-labeling bind mount option is ignored on platforms without SELinux. Note Relative host paths are only supported by Compose that deploy to a local container runtime. This is because the relative path is resolved from the Compose file\u2019s parent directory which is only applicable in the local case. When Compose deploys to a non-local platform it rejects Compose files which use relative host paths with an error. To avoid ambiguities with named volumes, relative paths should always begin with . or .. . Long syntax The long form syntax allows the configuration of additional fields that can't be expressed in the short form. type : The mount type. Either volume , bind , tmpfs , npipe , or cluster source : The source of the mount, a path on the host for a bind mount, or the name of a volume defined in the top-level volumes key . Not applicable for a tmpfs mount. target : The path in the container where the volume is mounted. read_only : Flag to set the volume as read-only. bind : Used to configure additional bind options: propagation : The propagation mode used for the bind. create_host_path : Creates a directory at the source path on host if there is nothing present. Compose does nothing if there is something present at the path. This is automatically implied by short syntax for backward compatibility with docker-compose legacy. selinux : The SELinux re-labeling option z (shared) or Z (private) volume : Configures additional volume options: nocopy : Flag to disable copying of data from a container when a volume is created. tmpfs : Configures additional tmpfs options: size : The size for the tmpfs mount in bytes (either numeric or as bytes unit). mode : The file mode for the tmpfs mount as Unix permission bits as an octal number. consistency : The consistency requirements of the mount. Available values are platform specific. volumes_from volumes_from mounts all of the volumes from another service or container. You can optionally specify read-only access ro or read-write rw . If no access level is specified, then read-write access is used. You can also mount volumes from a container that is not managed by Compose by using the container: prefix. volumes_from: - service_name - service_name:ro - container:container_name - container:container_name:rw working_dir working_dir overrides the container's working directory which is specified by the image, for example Dockerfile's WORKDIR . Networks top-level element Networks are the layer that allow services to communicate with each other. The top-level networks element lets you configure named networks that can be reused across multiple services. To use a network across multiple services, you must explicitly grant each service access by using the networks attribute within the services top-level element. The networks top-level element has additional syntax that provides more granular control. Examples Basic example In the following example, at runtime, networks front-tier and back-tier are created and the frontend service is connected to front-tier and back-tier networks. services: frontend: image: example/webapp networks: - front-tier - back-tier networks: front-tier: back-tier: Advanced example services: proxy: build: ./proxy networks: - frontend app: build: ./app networks: - frontend - backend db: image: postgres networks: - backend networks: frontend: ## Use a custom driver driver: custom-driver-1 backend: ## Use a custom driver which takes special options driver: custom-driver-2 driver_opts: foo: \"1\" bar: \"2\" The advanced example shows a Compose file which defines two custom networks. The proxy service is isolated from the db service, because they do not share a network in common. Only app can talk to both. Attributes driver driver specifies which driver should be used for this network. Compose returns an error if the driver is not available on the platform. networks: db-data: driver: overlay Default and available values are platform specific. Compose supports the following drivers: none and host host : Use the host's networking stack. none : Turn off networking. host or none The syntax for using built-in networks such as host and none is different, as such networks implicitly exist outside the scope of Compose. To use them, you must define an external network with the name host or none and an alias that Compose can use ( hostnet and nonet in the following example), then grant the service access to that network using its alias. services: web: networks: hostnet: {} networks: hostnet: external: true name: host services: web: ... networks: nonet: {} networks: nonet: external: true name: none driver_opts driver_opts specifies a list of options as key-value pairs to pass to the driver. These options are driver-dependent. Consult the driver's documentation for more information. networks: db-data: driver_opts: foo: \"bar\" baz: 1 attachable If attachable is set to true , then standalone containers should be able to attach to this network, in addition to services. If a standalone container attaches to the network, it can communicate with services and other standalone containers that are also attached to the network. networks: mynet1: driver: overlay attachable: true enable_ipv6 enable_ipv6 enables IPv6 networking. For an example, see step four of Create an IPv6 network . external If set to true : - external specifies that this network\u2019s lifecycle is maintained outside of that of the application. Compose doesn't attempt to create these networks, and returns an error if one doesn't exist. - All other attributes apart from name are irrelevant. If Compose detects any other attribute, it rejects the Compose file as invalid. In the example below, proxy is the gateway to the outside world. Instead of attempting to create a network, Compose queries the platform for an existing network simply called outside and connects the proxy service's containers to it. services: proxy: image: example/proxy networks: - outside - default app: image: example/app networks: - default networks: outside: external: true ipam ipam specifies a custom IPAM configuration. This is an object with several properties, each of which is optional: driver : Custom IPAM driver, instead of the default. config : A list with zero or more configuration elements, each containing a: subnet : Subnet in CIDR format that represents a network segment ip_range : Range of IPs from which to allocate container IPs gateway : IPv4 or IPv6 gateway for the master subnet aux_addresses : Auxiliary IPv4 or IPv6 addresses used by Network driver, as a mapping from hostname to IP options : Driver-specific options as a key-value mapping. networks: mynet1: ipam: driver: default config: - subnet: 172.28.0.0/16 ip_range: 172.28.5.0/24 gateway: 172.28.5.254 aux_addresses: host1: 172.28.1.5 host2: 172.28.1.6 host3: 172.28.1.7 options: foo: bar baz: \"0\" internal By default, Compose provides external connectivity to networks. internal , when set to true , allows you to create an externally isolated network. labels Add metadata to containers using labels . You can use either an array or a dictionary. It is recommended that you use reverse-DNS notation to prevent labels from conflicting with those used by other software. networks: mynet1: labels: com.example.description: \"Financial transaction network\" com.example.department: \"Finance\" com.example.label-with-empty-value: \"\" networks: mynet1: labels: - \"com.example.description=Financial transaction network\" - \"com.example.department=Finance\" - \"com.example.label-with-empty-value\" Compose sets com.docker.compose.project and com.docker.compose.network labels. name name sets a custom name for the network. The name field can be used to reference networks which contain special characters. The name is used as is and is not scoped with the project name. networks: network1: name: my-app-net It can also be used in conjunction with the external property to define the platform network that Compose should retrieve, typically by using a parameter so the Compose file doesn't need to hard-code runtime specific values: networks: network1: external: true name: \"${NETWORK_ID}\" Volumes top-level element Volumes are persistent data stores implemented by the container engine. Compose offers a neutral way for services to mount volumes, and configuration parameters to allocate them to infrastructure. The top-level volumes declaration lets you configure named volumes that can be reused across multiple services. To use a volume across multiple services, you must explicitly grant each service access by using the volumes attribute within the services top-level element. The volumes attribute has additional syntax that provides more granular control. Example The following example shows a two-service setup where a database's data directory is shared with another service as a volume, named db-data , so that it can be periodically backed up. services: backend: image: example/database volumes: - db-data:/etc/data backup: image: backup-service volumes: - db-data:/var/lib/backup/data volumes: db-data: The db-data volume is mounted at the /var/lib/backup/data and /etc/data container paths for backup and backend respectively. Running docker compose up creates the volume if it doesn't already exist. Otherwise, the existing volume is used and is recreated if it's manually deleted outside of Compose. Attributes An entry under the top-level volumes section can be empty, in which case it uses the container engine's default configuration for creating a volume. Optionally, you can configure it with the following keys: driver Specifies which volume driver should be used. Default and available values are platform specific. If the driver is not available, Compose returns an error and doesn't deploy the application. volumes: db-data: driver: foobar driver_opts driver_opts specifies a list of options as key-value pairs to pass to the driver for this volume. The options are driver-dependent. volumes: example: driver_opts: type: \"nfs\" o: \"addr=10.40.0.199,nolock,soft,rw\" device: \":/docker/example\" external If set to true : - external specifies that this volume already exists on the platform and its lifecycle is managed outside of that of the application. Compose doesn't then create the volume, and returns an error if the volume doesn't exist. - All other attributes apart from name are irrelevant. If Compose detects any other attribute, it rejects the Compose file as invalid. In the example below, instead of attempting to create a volume called {project_name}_db-data , Compose looks for an existing volume simply called db-data and mounts it into the backend service's containers. services: backend: image: example/database volumes: - db-data:/etc/data volumes: db-data: external: true labels labels are used to add metadata to volumes. You can use either an array or a dictionary. It's recommended that you use reverse-DNS notation to prevent your labels from conflicting with those used by other software. volumes: db-data: labels: com.example.description: \"Database volume\" com.example.department: \"IT/Ops\" com.example.label-with-empty-value: \"\" volumes: db-data: labels: - \"com.example.description=Database volume\" - \"com.example.department=IT/Ops\" - \"com.example.label-with-empty-value\" Compose sets com.docker.compose.project and com.docker.compose.volume labels. name name sets a custom name for a volume. The name field can be used to reference volumes that contain special characters. The name is used as is and is not scoped with the stack name. volumes: db-data: name: \"my-app-data\" This makes it possible to make this lookup name a parameter of the Compose file, so that the model ID for the volume is hard-coded but the actual volume ID on the platform is set at runtime during deployment. For example, if DATABASE_VOLUME=my_volume_001 in your .env file: volumes: db-data: name: ${DATABASE_VOLUME} Running docker compose up uses the volume called my_volume_001 . It can also be used in conjunction with the external property. This means the name of the volume used to lookup the actual volume on the platform is set separately from the name used to refer to it within the Compose file: volumes: db-data: external: name: actual-name-of-volume Configs top-level element Configs allow services to adapt their behaviour without the need to rebuild a Docker image. Services can only access configs when explicitly granted by a configs attribute within the services top-level element. As with volumes, configs are mounted as files into a service's container's filesystem. The location of the mount point within the container defaults to / in Linux containers and C:\\ in Windows containers. By default, the config: - Is owned by the user running the container command but can be overridden by service configuration. - Has world-readable permissions (mode 0444), unless the service is configured to override this. The top-level configs declaration defines or references configuration data that is granted to services in your Compose application. The source of the config is either file or external . file : The config is created with the contents of the file at the specified path. environment : The config content is created with the value of an environment variable. content : The content is created with the inlined value. external : If set to true, external specifies that this config has already been created. Compose does not attempt to create it, and if it does not exist, an error occurs. name : The name of the config object in the container engine to look up. This field can be used to reference configs that contain special characters. The name is used as is and will not be scoped with the project name. Example 1 _http_config is created when the application is deployed, by registering the content of the httpd.conf as the configuration data. configs: http_config: file: ./httpd.conf Alternatively, http_config can be declared as external. Compose looks up http_config to expose the configuration data to relevant services. configs: http_config: external: true Example 2 _app_config is created when the application is deployed, by registering the inlined content as the configuration data. This comes with the benefits Compose will infer variables when creating the config, which allows to adjust content according to service configuration: configs: app_config: content: | debug=${DEBUG} spring.application.admin.enabled=${DEBUG} spring.application.name=${COMPOSE_PROJECT_NAME} Example 3 External configs lookup can also use a distinct key by specifying a name . The following example modifies the previous one to look up a config using the parameter HTTP_CONFIG_KEY . The the actual lookup key will is set at deployment time by the interpolation of variables, but exposed to containers as hard-coded ID http_config . configs: http_config: external: true name: \"${HTTP_CONFIG_KEY}\" If external is set to true , all other attributes apart from name are irrelevant. If Compose detecs any other attribute, it rejects the Compose file as invalid. Secrets top-level element Secrets are a flavor of Configs focusing on sensitive data, with specific constraint for this usage. Services can only access secrets when explicitly granted by a secrets attribute within the services top-level element. The top-level secrets declaration defines or references sensitive data that is granted to the services in your Compose application. The source of the secret is either file or environment . file : The secret is created with the contents of the file at the specified path. environment : The secret is created with the value of an environment variable. external : If set to true, external specifies that this secret has already been created. Compose does not attempt to create it, and if it does not exist, an error occurs. name : The name of the secret object in Docker. This field can be used to reference secrets that contain special characters. The name is used as is and isn't scoped with the project name. Example 1 server-certificate secret is created as _server-certificate when the application is deployed, by registering content of the server.cert as a platform secret. secrets: server-certificate: file: ./server.cert Example 2 token secret is created as _token when the application is deployed, by registering the content of the OAUTH_TOKEN environment variable as a platform secret. secrets: token: environment: \"OAUTH_TOKEN\" Alternatively, server-certificate can be declared as external. Compose looks up the server-certificate secret to expose to relevant services. secrets: server-certificate: external: true Example 3 External secrets lookup can also use a distinct key by specifying a name . The following example modifies the previous example to look up a secret using the name CERTIFICATE_KEY . The actual lookup key is set at deployment time by the interpolation of variables, but exposed to containers as hard-coded ID server-certificate . secrets: server-certificate: external: true name: \"${CERTIFICATE_KEY}\" If external is set to true , all other attributes apart from name are irrelevant. If Compose detects any other attribute, it rejects the Compose file as invalid. Your Compose file needs to explicitly grant access to the secrets to relevant services in your application. Fragments With Compose, you can use built-in YAML features to make your Compose file neater and more efficient. Anchors and aliases let you create re-usable blocks. This is useful if you start to find common configurations that span multiple services. Having re-usable blocks minimizes potential mistakes. Anchors are created using the & sign. The sign is followed by an alias name. You can use this alias with the * sign later to reference the value following the anchor. Make sure there is no space between the & and the * characters and the following alias name. You can use more than one anchor and alias in a single Compose file. Example 1 volumes: db-data: &default-volume driver: default metrics: *default-volume In the example above, a default-volume anchor is created based on the db-data volume. It is later reused by the alias *default-volume to define the metrics volume. Anchor resolution takes place before variables interpolation , so variables can't be used to set anchors or aliases. Example 2 services: first: image: my-image:latest environment: &env - CONFIG_KEY - EXAMPLE_KEY - DEMO_VAR second: image: another-image:latest environment: *env If you have an anchor that you want to use in more than one service, use it in conjunction with an extension to make your Compose file easier to maintain. Example 3 You may want to partially override values. Compose follows the rule outlined by YAML merge type . In the following example, metrics volume specification uses alias to avoid repetition but overrides name attribute: services: backend: image: example/database volumes: - db-data - metrics volumes: db-data: &default-volume driver: default name: \"data\" metrics: <<: *default-volume name: \"metrics\" Example 4 You can also extend the anchor to add additional values. services: first: image: my-image:latest environment: &env FOO: BAR ZOT: QUIX second: image: another-image:latest environment: <<: *env YET_ANOTHER: VARIABLE Note YAML merge only applies to mappings, and can't be used with sequences. In example above, the environment variables must be declared using the FOO: BAR mapping syntax, while the sequence syntax - FOO=BAR is only valid when no fragments are involved. Extension As with Fragments , Extensions can be used to make your Compose file more efficient and easier to maintain. Extensions can also be used with anchors and aliases . Use the prefix x- as a top-level element to modularize configurations that you want to reuse. Compose ignores any fields that start with x- , this is the sole exception where Compose silently ignores unrecognized fields. They also can be used within any structure in a Compose file where user-defined keys are not expected. Compose use those to enable experimental features, the same way browsers add support for custom CSS features Example 1 x-custom: foo: - bar - zot services: webapp: image: example/webapp x-foo: bar service: backend: deploy: placement: x-aws-role: \"arn:aws:iam::XXXXXXXXXXXX:role/foo\" x-aws-region: \"eu-west-3\" x-azure-region: \"france-central\" Example 2 x-env: &env environment: - CONFIG_KEY - EXAMPLE_KEY services: first: <<: *env image: my-image:latest second: <<: *env image: another-image:latest In this example, the environment variables do not belong to either of the services. They\u2019ve been lifted out completely into the x-env extension field. This defines a new node which contains the environment field. The &env YAML anchor is used so both services can reference the extension field\u2019s value as *env . Example 3 x-function: &function labels: function: \"true\" depends_on: - gateway networks: - functions deploy: placement: constraints: - 'node.platform.os == linux' services: ## Node.js gives OS info about the node (Host) nodeinfo: <<: *function image: functions/nodeinfo:latest environment: no_proxy: \"gateway\" https_proxy: $https_proxy ## Uses `cat` to echo back response, fastest function to execute. echoit: <<: *function image: functions/alpine:health environment: fprocess: \"cat\" no_proxy: \"gateway\" https_proxy: $https_proxy The nodeinfo and echoit services both include the x-function extension via the &function anchor, then set their specific image and environment. Example 4 Using YAML merge it is also possible to use multiple extensions and share and override additional attributes for specific needs: x-environment: &default-environment FOO: BAR ZOT: QUIX x-keys: &keys KEY: VALUE services: frontend: image: example/webapp environment: << : [*default-environment, *keys] YET_ANOTHER: VARIABLE Note YAML merge only applies to mappings, and can't be used with sequences. In the example above, the environment variables are declared using the FOO: BAR mapping syntax, while the sequence syntax - FOO=BAR is only valid when no fragments are involved. Informative Historical Notes This section is informative. At the time of writing, the following prefixes are known to exist: Prefix Vendor/Organization docker Docker kubernetes Kubernetes Specifying byte values Values express a byte value as a string in {amount}{byte unit} format: The supported units are b (bytes), k or kb (kilo bytes), m or mb (mega bytes) and g or gb (giga bytes). 2b 1024kb 2048k 300m 1gb Specifying durations Values express a duration as a string in the form of {value}{unit} . The supported units are us (microseconds), ms (milliseconds), s (seconds), m (minutes) and h (hours). Values can combine multiple values without separator. 10ms 40s 1m30s 1h5m30s20ms Interpolation Values in a Compose file can be set by variables and interpolated at runtime. Compose files use a Bash-like syntax ${VARIABLE} . Both $VARIABLE and ${VARIABLE} syntax is supported. Default values can be defined inline using typical shell syntax: ${VARIABLE:-default} evaluates to default if VARIABLE is unset or empty in the environment. ${VARIABLE-default} evaluates to default only if VARIABLE is unset in the environment. Similarly, the following syntax allows you to specify mandatory variables: ${VARIABLE:?err} exits with an error message containing err if VARIABLE is unset or empty in the environment. ${VARIABLE?err} exits with an error message containing err only if VARIABLE is unset in the environment. Interpolation can also be nested: ${VARIABLE:-${FOO}} ${VARIABLE?$FOO} ${VARIABLE:-${FOO:-default}} Other extended shell-style features, such as ${VARIABLE/foo/bar} , are not supported by Compose. You can use a $$ (double-dollar sign) when your configuration needs a literal dollar sign. This also prevents Compose from interpolating a value, so a $$ allows you to refer to environment variables that you don't want processed by Compose. web: build: . command: \"$$VAR_NOT_INTERPOLATED_BY_COMPOSE\" If Compose can't resolve a substituted variable and no default value is defined, it displays a warning and substitutes the variable with an empty string. As any values in a Compose file can be interpolated with variable substitution, including compact string notation for complex elements, interpolation is applied before a merge on a per-file basis. Interpolation applies only to YAML values, not to keys. For the few places where keys are actually arbitrary user-defined strings, such as labels or environment , an alternate equal sign syntax must be used for interpolation to apply. For example: services: foo: labels: \"$VAR_NOT_INTERPOLATED_BY_COMPOSE\": \"BAR\" services: foo: labels: - \"$VAR_INTERPOLATED_BY_COMPOSE=BAR\" Merge and override Compose lets you define a Compose application model through multiple Compose files . When doing so, Compose follows the rules declared in this section to merge Compose files. Mapping A YAML mapping gets merged by adding missing entries and merging the conflicting ones. Merging the following example YAML trees: services: foo: key1: value1 key2: value2 services: foo: key2: VALUE key3: value3 Results in a Compose application model equivalent to the YAML tree: services: foo: key1: value1 key2: VALUE key3: value3 Sequence A YAML sequence is merged by appending values from the overriding Compose file to the previous one. Merging the following example YAML trees: services: foo: DNS: - 1.1.1.1 services: foo: DNS: - 8.8.8.8 Results in a Compose application model equivalent to the YAML tree: services: foo: DNS: - 1.1.1.1 - 8.8.8.8 Exceptions Shell commands When merging Compose files that use the services attributes command , entrypoint and healthcheck: test , the value is overridden by the latest Compose file, and not appended. Merging the following example YAML trees: services: foo: command: [\"echo\", \"foo\"] services: foo: command: [\"echo\", \"bar\"] Results in a Compose application model equivalent to the YAML tree: services: foo: command: [\"echo\", \"bar\"] Unique resources Applies to the ports , volumes , secrets and configs services attributes. While these types are modeled in a Compose file as a sequence, they have special uniqueness requirements: Attribute Unique key volumes target secrets source configs source ports {ip, target, published, protocol} When merging Compose files, Compose appends new entries that do not violate a uniqueness constraint and merge entries that share a unique key. Merging the following example YAML trees: services: foo: volumes: - foo:/work services: foo: volumes: - bar:/work Results in a Compose application model equivalent to the YAML tree: services: foo: volumes: - bar:/work Reset value In addition to the previously described mechanism, an override Compose file can also be used to remove elements from your application model. For this purpose, the custom YAML tag !reset can be set to override a value set by the overriden Compose file. A valid value for attribute must be provided, but will be ignored and target attribute will be set with type's default value or null . For readability, it is recommended to explicitly set the attribute value to the null ( null ) or empty array [] (with !reset null or !reset [] ) so that it is clear that resulting attribute will be cleared. A base compose.yaml file: services: app: image: myapp ports: - \"8080:80\" environment: FOO: BAR And an overide.compose.yaml file: services: app: image: myapp ports: !reset [] environment: FOO: !reset null Results in: services: app: image: myapp Replace value While !reset can be used to remove a declaration from a Compose file using an override file, !override allows you to fully replace an attribute, bypassing the standard merge rules. A typical example is to fully replace a resource definition, to rely on a distinct model but using the same name. A base compose.yaml file: services: app: image: myapp ports: - \"8080:80\" To remove the original port, but expose a new one, the following override file is used: services: app: ports: !override - \"8443:443\" This results in: services: app: image: myapp ports: - \"8443:443\" If !override had not been used, both 8080:80 and 8443:443 would be exposed as per the merging rules outlined above . Include A Compose application can declare dependency on another Compose application. This is useful if: - You want to reuse other Compose files. - You need to factor out parts of your application model into separate Compose files so they can be managed separately or shared with others. - Teams need to keep a Compose file reasonably complicated for the limited amount of resources it has to declare for it's own sub-domain, within a larger deployment. The include top-level section is used to define the dependency on another Compose application, or sub-domain. Each path listed in the include section is loaded as an individual Compose application model, with it's own project directory, in order to resolve relative paths. Once the included Compose application is loaded, all resources definitions are copied into the current Compose application model. Compose displays a warning if resource names conflict and doesn't try to merge them. To enforce this, include is evaluated after the Compose file(s) selected to define the Compose application model have been parsed and merged, so that conflicts between Compose files are detected. include applies recursively so an included Compose file which declares its own include section, triggers those other files to be included as well. Any volumes, networks, or other resources pulled in from the included Compose file can be used by the current Compose application for cross-service references. For example: include: - my-compose-include.yaml #with serviceB declared services: serviceA: build: . depends_on: - serviceB #use serviceB directly as if it was declared in this Compose file Compose also supports the use of interpolated variables with include . It's recommended that you specify mandatory variables . For example: include: -${INCLUDE_PATH:?FOO}/compose.yaml Short syntax The short syntax only defines paths to other Compose files. The file is loaded with the parent folder as the project directory, and an optional .env file that is loaded to define any variables' default values by interpolation. The local project's environment can override those values. include: - ../commons/compose.yaml - ../another_domain/compose.yaml services: webapp: depends_on: - included-service ## defined by another_domain In the above example, both ../commons/compose.yaml and ../another_domain/compose.yaml are loaded as individual Compose projects. Relative paths in Compose files being referred by include are resolved relative to their own Compose file path, not based on the local project's directory. Variables are interpolated using values set in the optional .env file in same folder, and is overridden by the local project's environment. Long syntax The long syntax offers more control over the sub-project parsing: include: - path: ../commons/compose.yaml project_directory: .. env_file: ../another/.env path path is required and defines the location of the Compose file(s) to be parsed and included into the local Compose model. path can be set either to a string when a single Compose file is involved, or to a list of strings when multiple Compose files need to be merged together to define the Compose model to be included in the local application. include: - path: - ../commons/compose.yaml - ./commons-override.yaml project_directory project_directory defines a base path to resolve relative paths set in the Compose file. It defaults to the directory of the included Compose file. env_file env_file defines an environment file(s) to use to define default values when interpolating variables in the Compose file being parsed. It defaults to .env file in the project_directory for the Compose file being parsed. env_file can be set either to a string or a list of strings when multiple environment files need to be merged to define a project environment. include: - path: ../another/compose.yaml env_file: - ../another/.env - ../another/dev.env The local project's environment has precedence over the values set by the Compose file, so that the local project can override values for customization. Profiles With profiles you can define a set of active profiles so your Compose application model is adjusted for various usages and environments. The exact mechanism is implementation specific and may include command line flags, environment variables, etc. The services top-level element supports a profiles attribute to define a list of named profiles. Services without a profiles attribute are always enabled. A service is ignored by Compose when none of the listed profiles match the active ones, unless the service is explicitly targeted by a command. In that case its profile is added to the set of active profiles. Note All other top-level elements are not affected by profiles and are always active. References to other services (by links , extends or shared resource syntax service:xxx ) do not automatically enable a component that would otherwise have been ignored by active profiles. Instead Compose returns an error. Illustrative example services: foo: image: foo bar: image: bar profiles: - test baz: image: baz depends_on: - bar profiles: - test zot: image: zot depends_on: - bar profiles: - debug In the above example: If the Compose application model is parsed with no profile enabled, it only contains the foo service. If the profile test is enabled, the model contains the services bar and baz , and service foo , which is always enabled. If the profile debug is enabled, the model contains both foo and zot services, but not bar and baz , and as such the model is invalid regarding the depends_on constraint of zot . If the profiles debug and test are enabled, the model contains all services; foo , bar , baz and zot . If Compose is executed with bar as the explicit service to run, bar and the test profile are active even if test profile is not enabled. If Compose is executed with baz as the explicit service to run, the service baz and the profile test are active and bar is pulled in by the depends_on constraint. If Compose is executed with zot as the explicit service to run, again the model is invalid regarding the depends_on constraint of zot , since zot and bar have no common profiles listed. If Compose is executed with zot as the explicit service to run and profile test is enabled, profile debug is automatically enabled and service bar is pulled in as a dependency starting both services zot and bar . See how you can use profiles in Docker Compose .","title":"Docker"},{"location":"docker/#docker","text":"sudo apt update && sudo apt upgrade -y sudo apt install git curl htop mc rsync zsh-autosuggestions sudo apt install zsh chsh -s $(which zsh) sh -c \"$(curl -fsSL https://raw.githubusercontent.com/ohmyzsh/ohmyzsh/master/tools/install.sh)\" rsync -avuPz ./.oh-my-zsh/custom/themes/custom.zsh-theme furyhawk@arm:/home/furyhawk/.oh-my-zsh/custom/themes/custom.zsh-theme git clone https://github.com/zsh-users/zsh-autosuggestions.git $ZSH_CUSTOM/plugins/zsh-autosuggestions git clone https://github.com/zdharma-continuum/fast-syntax-highlighting.git ${ZSH_CUSTOM:-$HOME/.oh-my-zsh/custom}/plugins/fast-syntax-highlighting ## Enable plugins by adding them to .zshrc. - Open .zshrc `nano ~/.zshrc` - ZSH_THEME=\"custom\" - Find the line which says `plugins=(git)`. - Replace that line with `plugins=(git zsh-autosuggestions zsh-syntax-highlighting fast-syntax-highlighting zsh-autocomplete)` touch .zprofile nano .zprofile # set PATH so it includes user's private bin if it exists if [ -d \"$HOME/bin\" ] ; then PATH=\"$HOME/bin:$PATH\" fi # set PATH so it includes user's private bin if it exists if [ -d \"$HOME/.local/bin\" ] ; then PATH=\"$HOME/.local/bin:$PATH\" fi export TZ=Asia/Singapore export NODE_ID=$(docker info -f '{{.Swarm.NodeID}}') export EMAIL=furyx@hotmail.com export DOMAIN=furyhawk.lol sudo nano /etc/fstab touch .credentials nano .credentials sudo curl -fsSL https://download.docker.com/linux/debian/gpg -o /etc/apt/keyrings/docker.asc sudo chmod a+r /etc/apt/keyrings/docker.asc echo \\ \"deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.asc] https://download.docker.com/linux/debian \\ $(. /etc/os-release && echo \"$VERSION_CODENAME\") stable\" | \\ sudo tee /etc/apt/sources.list.d/docker.list > /dev/null sudo apt-get update sudo apt-get install docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin # sudo groupadd docker sudo usermod -aG docker $USER sudo systemctl enable docker.service sudo systemctl enable containerd.service systemctl list-units --type=service --state=active cd /etc/docker sudo touch daemon.json sudo nano daemon.json { \"log-driver\": \"json-file\", \"log-opts\": { \"max-size\": \"10m\", \"max-file\": \"3\" } } docker swarm join --token SWMTKN-1-xxx 192.168.50.114:2377 docker node update --availability drain node docker node update --availability Active node curl --silent --remote-name --location https://github.com/ceph/ceph/raw/octopus/src/cephadm/cephadm chmod +x cephadm sudo ./cephadm add-repo --release octopus sudo ./cephadm install cephadm install ceph-common mkdir -p /etc/ceph sudo ./cephadm bootstrap --mon-ip 192.168.65.19 URL: https://debian.local:8443/ User: admin Password: 12345678 You can access the Ceph CLI with: sudo ./cephadm shell --fsid 8ad7deb6-265a-11ef-9e81-c2d0c41fc7e0 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring Please consider enabling telemetry to help improve Ceph: ceph telemetry on For more information see: https://docs.ceph.com/docs/master/mgr/telemetry/ alias ceph='./cephadm shell -- ceph' sudo ceph -v ceph version 16.2.11 (3cf40e2dca667f68c6ce3ff5cd94f01e711af894) pacific (stable) sudo ceph orch host label add debian mon Added label mon to host debian sudo ceph orch apply mon debian Scheduled mon update... sudo ceph orch host ls ssh-copy-id -f -i /etc/ceph/ceph.pub root@ceph-2 ssh-copy-id -f -i /etc/ceph/ceph.pub osd1 ssh-copy-id -f -i /etc/ceph/ceph.pub osd2 ssh-copy-id -f -i /etc/ceph/ceph.pub osd3 sudo ceph orch host add osd1 sudo ceph orch host add osd2 sudo ceph orch host add osd3 sudo ceph orch apply osd --all-available-devices sudo ceph status sudo find / -iname 'cephadm*' 2>/dev/null systemctl --user start docker-desktop sudo groupadd docker sudo usermod -aG docker $USER groups ${USER} curl -fsSL test.docker.com -o get-docker.sh && sh get-docker.sh sudo apt-get install libffi-dev libssl-dev sudo apt install python3-dev sudo apt-get install -y python3 python3-pip pip install \"cython<3.0.0\" wheel pip install \"pyyaml==5.4.1\" --no-build-isolation pip install docker-compose sudo systemctl enable docker sudo apt-get purge docker-ce sudo apt-get purge docker-ce-cli sudo rm -rf /var/lib/docker rootless sudo apt install -y dbus-user-session sudo apt install -y fuse-overlayfs sudo apt install -y fuse-overlayfs sudo systemctl disable --now docker.service docker.socket sudo rm -rf /var/lib/docker sudo apt-get purge docker-ce docker-ce-cli containerd.io sudo apt-get purge docker-ce docker-ce-cli containerd.io apt-get install -y uidmap curl -fsSL https://get.docker.com/rootless | sh nano .zshrc export PATH=/home/furyhawk/bin:$PATH export DOCKER_HOST=unix:///run/user/1000/docker.sock systemctl --user enable docker sudo loginctl enable-linger $(whoami) docker context use rootless sudo setcap cap_net_bind_service=ep $(which rootlesskit) systemctl --user restart docker #sudo apt-get install -y docker-ce-rootless-extras dockerd-rootless-setuptool.sh uninstall cd ~/bin rm -f containerd containerd-shim containerd-shim-runc-v2 ctr docker docker-init docker-proxy dockerd dockerd-rootless-setuptool.sh dockerd-rootless.sh rootlesskit rootlesskit-docker-proxy runc vpnkit netstat -ltup","title":"Docker"},{"location":"docker/#the-compose-specification","text":"{:.no_toc} ToC {:toc}","title":"The Compose Specification"},{"location":"docker/#status-of-this-document","text":"This document specifies the Compose file format used to define multi-containers applications. Distribution of this document is unlimited.","title":"Status of this document"},{"location":"docker/#requirements-and-optional-attributes","text":"The Compose specification includes properties designed to target a local OCI container runtime, exposing Linux kernel specific configuration options, but also some Windows container specific properties. It is also designed for cloud platform features related to resource placement on a cluster, replicated application distribution, and scalability. We acknowledge that no Compose implementation is expected to support all attributes, and that support for some properties is platform dependent and can only be confirmed at runtime. The definition of a versioned schema to control the supported properties in a Compose file, established by the docker-compose tool where the Compose file format was designed, doesn't offer any guarantee to the end-user that attributes will be actually implemented. The specification defines the expected configuration syntax and behavior. Unless noted, supporting any of these is optional. A Compose implementation to parse a Compose file using unsupported attributes should warn users. We recommend the following implementors to support those running modes: Default: warn the user about unsupported attributes, but ignore them Strict: warn the user about unsupported attributes and reject the Compose file Loose: ignore unsupported attributes AND unknown attributes (that were not defined by the spec by the time implementation was created) From this point onwards, references made to 'Compose' can be interpreted as 'a Compose implementation'.","title":"Requirements and optional attributes"},{"location":"docker/#the-compose-application-model","text":"The Compose Specification lets you define a platform-agnostic container based application. Such an application is designed as a set of containers which have to both run together with adequate shared resources and communication channels. Computing components of an application are defined as services . A service is an abstract concept implemented on platforms by running the same container image, and configuration, one or more times. Services communicate with each other through networks . In the Compose Specification, a network is a platform capability abstraction to establish an IP route between containers within services connected together. Low-level, platform-specific networking options are grouped into the Network definition and may be partially implemented on some platforms. Services store and share persistent data into volumes . The Specification describes such a persistent data as a high-level filesystem mount with global options. Actual platform-specific implementation details are grouped into the volumes definition and may be partially implemented on some platforms. Some services require configuration data that is dependent on the runtime or platform. For this, the Specification defines a dedicated configs concept. From a service container point of view, configs are comparable to volumes, in that they are files mounted into the container. But the actual definition involves distinct platform resources and services, which are abstracted by this type. A secret is a specific flavor of configuration data for sensitive data that should not be exposed without security considerations. Secrets are made available to services as files mounted into their containers, but the platform-specific resources to provide sensitive data are specific enough to deserve a distinct concept and definition within the Compose specification. Note With volumes, configs and secrets you can have a simple declaration at the top-level and then add more platform-specific information at the service level. A project is an individual deployment of an application specification on a platform. A project's name, set with the top-level name attribute, is used to group resources together and isolate them from other applications or other installation of the same Compose specified application with distinct parameters. If you are creating resources on a platform, you must prefix resource names by project and set the label com.docker.compose.project . Compose offers a way for users to set a custom project name and override this name, so that the same compose.yaml file can be deployed twice on the same infrastructure, without changes, by just passing a distinct name. Project names must contain only lowercase letters, decimal digits, dashes, and underscores, and must begin with a lowercase letter or decimal digit.","title":"The Compose application model"},{"location":"docker/#illustrative-example","text":"The following example illustrates the Compose Specification concepts outlined above. The example is non-normative. Consider an application split into a frontend web application and a backend service. The frontend is configured at runtime with an HTTP configuration file managed by infrastructure, providing an external domain name, and an HTTPS server certificate injected by the platform's secured secret store. The backend stores data in a persistent volume. Both services communicate with each other on an isolated back-tier network, while the frontend is also connected to a front-tier network and exposes port 443 for external usage. %%{ init: { 'flowchart': { 'curve': 'linear' } } }%% flowchart LR subgraph A[INFRASTRUCTURE] direction TB subgraph TOP[\" \"] subgraph B1[Frontend Service] fs[\"`**webapp**`\"] end style B1 fill:#ccd6e8, stroke-width:0px subgraph B2[Backend Service] bs[\"`**database**`\"] end style B2 fill:#ccd6e8, stroke-width:0px end style TOP fill:transparent, stroke-width:2px, stroke:#62affb, stroke-dasharray: 5 5 key[ro= read only\\nr+w = read write] style key fill:transparent, stroke-width:0px,text-align: left, size: 94px direction TB id2(Server\\nCertificate) id1(HTTP\\nConfiguration) id1 & id2 -.-|ro| B1 style id1 stroke:#000,stroke-width:1px,stroke-dasharray: 10 style id2 stroke:#000,stroke-width:1px,stroke-dasharray: 10 B2 ==r+w==> id3[(Persistent\\nVolume)] end style A fill:#eeeeee, stroke-width:0px direction LR id4[External\\nUser] ---id5(((443)))--->|Frontend\\nNetwork| B1 style id4 stroke:#000,stroke-width:2px B1 --Backend\\nNetwork--> B2 The example application is composed of the following parts: 2 services, backed by Docker images: webapp and database 1 secret (HTTPS certificate), injected into the frontend 1 configuration (HTTP), injected into the frontend 1 persistent volume, attached to the backend 2 networks services: frontend: image: example/webapp ports: - \"443:8043\" networks: - front-tier - back-tier configs: - httpd-config secrets: - server-certificate backend: image: example/database volumes: - db-data:/etc/data networks: - back-tier volumes: db-data: driver: flocker driver_opts: size: \"10GiB\" configs: httpd-config: external: true secrets: server-certificate: external: true networks: ## The presence of these objects is sufficient to define them front-tier: {} back-tier: {} This example illustrates the distinction between volumes, configs and secrets. While all of them are all exposed to service containers as mounted files or directories, only a volume can be configured for read+write access. Secrets and configs are read-only. The volume configuration allows you to select a volume driver and pass driver options to tweak volume management according to the actual infrastructure. Configs and secrets rely on platform services, and are declared external as they are not managed as part of the application lifecycle. Compose uses a platform-specific lookup mechanism to retrieve runtime values.","title":"Illustrative example"},{"location":"docker/#compose-file","text":"The Compose file is a YAML file defining: - Version (Optional) - Services (Required) - Networks - Volumes - Configs - Secrets The default path for a Compose file is compose.yaml (preferred) or compose.yml that is placed in the working directory. Compose also supports docker-compose.yaml and docker-compose.yml for backwards compatibility of earlier versions. If both files exist, Compose prefers the canonical compose.yaml . You can use fragments and extensions to keep your Compose file efficient and easy to maintain. Multiple Compose files can be merged together to define the application model. The combination of YAML files are implemented by appending or overriding YAML elements based on the Compose file order you set. Simple attributes and maps get overridden by the highest order Compose file, lists get merged by appending. Relative paths are resolved based on the first Compose file's parent folder, whenever complimentary files being merged are hosted in other folders. As some Compose file elements can both be expressed as single strings or complex objects, merges apply to the expanded form. If you want to reuse other Compose files, or factor out parts of you application model into separate Compose files, you can also use include . This is useful if your Compose application is dependent on another application which is managed by a different team, or needs to be shared with others.","title":"Compose file"},{"location":"docker/#version-and-name-top-level-elements","text":"","title":"Version and name top-level elements"},{"location":"docker/#version-top-level-element","text":"The top-level version property is defined by the Compose Specification for backward compatibility. It is only informative. Compose doesn't use version to select an exact schema to validate the Compose file, but prefers the most recent schema when it's implemented. Compose validates whether it can fully parse the Compose file. If some fields are unknown, typically because the Compose file was written with fields defined by a newer version of the Specification, you'll receive a warning message. Compose offers options to ignore unknown fields (as defined by \"loose\" mode).","title":"Version top-level element"},{"location":"docker/#name-top-level-element","text":"The top-level name property is defined by the Specification as the project name to be used if you don't set one explicitly. Compose offers a way for you to override this name, and sets a default project name to be used if the top-level name element is not set. Whenever a project name is defined by top-level name or by some custom mechanism, it is exposed for interpolation and environment variable resolution as COMPOSE_PROJECT_NAME services: foo: image: busybox environment: - COMPOSE_PROJECT_NAME command: echo \"I'm running ${COMPOSE_PROJECT_NAME}\"","title":"Name top-level element"},{"location":"docker/#services-top-level-element","text":"A service is an abstract definition of a computing resource within an application which can be scaled or replaced independently from other components. Services are backed by a set of containers, run by the platform according to replication requirements and placement constraints. As services are backed by containers, they are defined by a Docker image and set of runtime arguments. All containers within a service are identically created with these arguments. A Compose file must declare a services top-level element as a map whose keys are string representations of service names, and whose values are service definitions. A service definition contains the configuration that is applied to each service container. Each service may also include a build section, which defines how to create the Docker image for the service. Compose supports building docker images using this service definition. If not used, the build section is ignored and the Compose file is still considered valid. Build support is an optional aspect of the Compose Specification, and is described in detail in the Compose Build Specification documentation. Each service defines runtime constraints and requirements to run its containers. The deploy section groups these constraints and allows the platform to adjust the deployment strategy to best match containers' needs with available resources. Deploy support is an optional aspect of the Compose Specification, and is described in detail in the Compose Deploy Specification documentation. If not implemented the deploy section is ignored and the Compose file is still considered valid.","title":"Services top-level element"},{"location":"docker/#attach","text":"When attach is defined and set to false Compose does not collect service logs, until you explicitly request it to. The default service configuration is attach: true .","title":"attach"},{"location":"docker/#build","text":"build specifies the build configuration for creating a container image from source, as defined in the Compose Build Specification .","title":"build"},{"location":"docker/#blkio_config","text":"blkio_config defines a set of configuration options to set block IO limits for a service. services: foo: image: busybox blkio_config: weight: 300 weight_device: - path: /dev/sda weight: 400 device_read_bps: - path: /dev/sdb rate: '12mb' device_read_iops: - path: /dev/sdb rate: 120 device_write_bps: - path: /dev/sdb rate: '1024k' device_write_iops: - path: /dev/sdb rate: 30","title":"blkio_config"},{"location":"docker/#device_read_bps-device_write_bps","text":"Set a limit in bytes per second for read / write operations on a given device. Each item in the list must have two keys: path : Defines the symbolic path to the affected device. rate : Either as an integer value representing the number of bytes or as a string expressing a byte value.","title":"device_read_bps, device_write_bps"},{"location":"docker/#device_read_iops-device_write_iops","text":"Set a limit in operations per second for read / write operations on a given device. Each item in the list must have two keys: path : Defines the symbolic path to the affected device. rate : As an integer value representing the permitted number of operations per second.","title":"device_read_iops, device_write_iops"},{"location":"docker/#weight","text":"Modify the proportion of bandwidth allocated to a service relative to other services. Takes an integer value between 10 and 1000, with 500 being the default.","title":"weight"},{"location":"docker/#weight_device","text":"Fine-tune bandwidth allocation by device. Each item in the list must have two keys: path : Defines the symbolic path to the affected device. weight : An integer value between 10 and 1000.","title":"weight_device"},{"location":"docker/#cpu_count","text":"cpu_count defines the number of usable CPUs for service container.","title":"cpu_count"},{"location":"docker/#cpu_percent","text":"cpu_percent defines the usable percentage of the available CPUs.","title":"cpu_percent"},{"location":"docker/#cpu_shares","text":"cpu_shares defines, as integer value, a service container's relative CPU weight versus other containers.","title":"cpu_shares"},{"location":"docker/#cpu_period","text":"cpu_period configures CPU CFS (Completely Fair Scheduler) period when a platform is based on Linux kernel.","title":"cpu_period"},{"location":"docker/#cpu_quota","text":"cpu_quota configures CPU CFS (Completely Fair Scheduler) quota when a platform is based on Linux kernel.","title":"cpu_quota"},{"location":"docker/#cpu_rt_runtime","text":"cpu_rt_runtime configures CPU allocation parameters for platforms with support for realtime scheduler. It can be either an integer value using microseconds as unit or a duration . cpu_rt_runtime: '400ms' cpu_rt_runtime: 95000`","title":"cpu_rt_runtime"},{"location":"docker/#cpu_rt_period","text":"cpu_rt_period configures CPU allocation parameters for platforms with support for realtime scheduler. It can be either an integer value using microseconds as unit or a duration . cpu_rt_period: '1400us' cpu_rt_period: 11000`","title":"cpu_rt_period"},{"location":"docker/#cpus","text":"DEPRECATED: use deploy.limits.cpus cpus define the number of (potentially virtual) CPUs to allocate to service containers. This is a fractional number. 0.000 means no limit.","title":"cpus"},{"location":"docker/#cpuset","text":"cpuset defines the explicit CPUs in which to allow execution. Can be a range 0-3 or a list 0,1","title":"cpuset"},{"location":"docker/#cap_add","text":"cap_add specifies additional container capabilities as strings. cap_add: - ALL","title":"cap_add"},{"location":"docker/#cap_drop","text":"cap_drop specifies container capabilities to drop as strings. cap_drop: - NET_ADMIN - SYS_ADMIN","title":"cap_drop"},{"location":"docker/#cgroup","text":"cgroup specifies the cgroup namespace to join. When unset, it is the container runtime's decision to select which cgroup namespace to use, if supported. host : Runs the container in the Container runtime cgroup namespace. private : Runs the container in its own private cgroup namespace.","title":"cgroup"},{"location":"docker/#cgroup_parent","text":"cgroup_parent specifies an optional parent cgroup for the container. cgroup_parent: m-executor-abcd","title":"cgroup_parent"},{"location":"docker/#command","text":"command overrides the default command declared by the container image, for example by Dockerfile's CMD . command: bundle exec thin -p 3000 The value can also be a list, in a manner similar to Dockerfile : command: [ \"bundle\", \"exec\", \"thin\", \"-p\", \"3000\" ] If the value is null , the default command from the image is used. If the value is [] (empty list) or '' (empty string), the default command declared by the image is ignored, i.e. overridden to be empty.","title":"command"},{"location":"docker/#configs","text":"Configs allow services to adapt their behaviour without the need to rebuild a Docker image. Services can only access configs when explicitly granted by the configs attribute. Two different syntax variants are supported. Compose reports an error if config doesn't exist on the platform or isn't defined in the configs top-level element in the Compose file. There are two syntaxes defined for configs. To remain compliant to this specification, an implementation must support both syntaxes. Implementations must allow use of both short and long syntaxes within the same document. You can grant a service access to multiple configs, and you can mix long and short syntax.","title":"configs"},{"location":"docker/#short-syntax","text":"The short syntax variant only specifies the config name. This grants the container access to the config and mounts it as files into a service\u2019s container\u2019s filesystem. The location of the mount point within the container defaults to / in Linux containers, and C:\\ in Windows containers. The following example uses the short syntax to grant the redis service access to the my_config and my_other_config configs. The value of my_config is set to the contents of the file ./my_config.txt , and my_other_config is defined as an external resource, which means that it has already been defined in the platform. If the external config does not exist, the deployment fails. services: redis: image: redis:latest configs: - my_config - my_other_config configs: my_config: file: ./my_config.txt my_other_config: external: true","title":"Short syntax"},{"location":"docker/#long-syntax","text":"The long syntax provides more granularity in how the config is created within the service's task containers. source : The name of the config as it exists in the platform. target : The path and name of the file to be mounted in the service's task containers. Defaults to / if not specified. uid and gid : The numeric UID or GID that owns the mounted config file within the service's task containers. Default value when not specified is USER running container. mode : The permissions for the file that is mounted within the service's task containers, in octal notation. Default value is world-readable ( 0444 ). Writable bit must be ignored. The executable bit can be set. The following example sets the name of my_config to redis_config within the container, sets the mode to 0440 (group-readable) and sets the user and group to 103 . The redis service does not have access to the my_other_config config. services: redis: image: redis:latest configs: - source: my_config target: /redis_config uid: \"103\" gid: \"103\" mode: 0440 configs: my_config: external: true my_other_config: external: true","title":"Long syntax"},{"location":"docker/#container_name","text":"container_name is a string that specifies a custom container name, rather than a name generated by default. container_name: my-web-container Compose does not scale a service beyond one container if the Compose file specifies a container_name . Attempting to do so results in an error. container_name follows the regex format of [a-zA-Z0-9][a-zA-Z0-9_.-]+","title":"container_name"},{"location":"docker/#credential_spec","text":"credential_spec configures the credential spec for a managed service account. If you have services that use Windows containers, you can use file: and registry: protocols for credential_spec . Compose also supports additional protocols for custom use-cases. The credential_spec must be in the format file:// or registry:// . credential_spec: file: my-credential-spec.json When using registry: , the credential spec is read from the Windows registry on the daemon's host. A registry value with the given name must be located in: HKLM\\SOFTWARE\\Microsoft\\Windows NT\\CurrentVersion\\Virtualization\\Containers\\CredentialSpecs The following example loads the credential spec from a value named my-credential-spec in the registry: credential_spec: registry: my-credential-spec","title":"credential_spec"},{"location":"docker/#example-gmsa-configuration","text":"When configuring a gMSA credential spec for a service, you only need to specify a credential spec with config , as shown in the following example: services: myservice: image: myimage:latest credential_spec: config: my_credential_spec configs: my_credentials_spec: file: ./my-credential-spec.json|","title":"Example gMSA configuration"},{"location":"docker/#depends_on","text":"depends_on expresses startup and shutdown dependencies between services.","title":"depends_on"},{"location":"docker/#short-syntax_1","text":"The short syntax variant only specifies service names of the dependencies. Service dependencies cause the following behaviors: Compose creates services in dependency order. In the following example, db and redis are created before web . Compose removes services in dependency order. In the following example, web is removed before db and redis . Simple example: services: web: build: . depends_on: - db - redis redis: image: redis db: image: postgres Compose guarantees dependency services have been started before starting a dependent service. Compose waits for dependency services to be \"ready\" before starting a dependent service.","title":"Short syntax"},{"location":"docker/#long-syntax_1","text":"The long form syntax enables the configuration of additional fields that can't be expressed in the short form. restart : When set to true Compose restarts this service after it updates the dependency service. This applies to an explicit restart controlled by a Compose operation, and excludes automated restart by the container runtime after the container dies. condition : Sets the condition under which dependency is considered satisfied service_started : An equivalent of the short syntax described above service_healthy : Specifies that a dependency is expected to be \"healthy\" (as indicated by healthcheck ) before starting a dependent service. service_completed_successfully : Specifies that a dependency is expected to run to successful completion before starting a dependent service. required : When set to false Compose only warns you when the dependency service isn't started or available. If it's not defined the default value of required is true . Service dependencies cause the following behaviors: Compose creates services in dependency order. In the following example, db and redis are created before web . Compose waits for healthchecks to pass on dependencies marked with service_healthy . In the following example, db is expected to be \"healthy\" before web is created. Compose removes services in dependency order. In the following example, web is removed before db and redis . services: web: build: . depends_on: db: condition: service_healthy restart: true redis: condition: service_started redis: image: redis db: image: postgres Compose guarantees dependency services are started before starting a dependent service. Compose guarantees dependency services marked with service_healthy are \"healthy\" before starting a dependent service.","title":"Long syntax"},{"location":"docker/#deploy","text":"deploy specifies the configuration for the deployment and lifecycle of services, as defined in the Compose Deploy Specification .","title":"deploy"},{"location":"docker/#develop","text":"develop specifies the development configuration for maintaining a container in sync with source, as defined in the Development Section .","title":"develop"},{"location":"docker/#device_cgroup_rules","text":"device_cgroup_rules defines a list of device cgroup rules for this container. The format is the same format the Linux kernel specifies in the Control Groups Device Whitelist Controller . device_cgroup_rules: - 'c 1:3 mr' - 'a 7:* rmw'","title":"device_cgroup_rules"},{"location":"docker/#devices","text":"devices defines a list of device mappings for created containers in the form of HOST_PATH:CONTAINER_PATH[:CGROUP_PERMISSIONS] . devices: - \"/dev/ttyUSB0:/dev/ttyUSB0\" - \"/dev/sda:/dev/xvda:rwm\"","title":"devices"},{"location":"docker/#dns","text":"dns defines custom DNS servers to set on the container network interface configuration. It can be a single value or a list. dns: 8.8.8.8 dns: - 8.8.8.8 - 9.9.9.9","title":"dns"},{"location":"docker/#dns_opt","text":"dns_opt list custom DNS options to be passed to the container\u2019s DNS resolver ( /etc/resolv.conf file on Linux). dns_opt: - use-vc - no-tld-query","title":"dns_opt"},{"location":"docker/#dns_search","text":"dns_search defines custom DNS search domains to set on container network interface configuration. It can be a single value or a list. dns_search: example.com dns_search: - dc1.example.com - dc2.example.com","title":"dns_search"},{"location":"docker/#domainname","text":"domainname declares a custom domain name to use for the service container. It must be a valid RFC 1123 hostname.","title":"domainname"},{"location":"docker/#entrypoint","text":"entrypoint declares the default entrypoint for the service container. This overrides the ENTRYPOINT instruction from the service's Dockerfile. If entrypoint is non-null, Compose ignores any default command from the image, for example the CMD instruction in the Dockerfile. See also command to set or override the default command to be executed by the entrypoint process. In its short form, the value can be defined as a string: entrypoint: /code/entrypoint.sh Alternatively, the value can also be a list, in a manner similar to the Dockerfile : entrypoint: - php - -d - zend_extension=/usr/local/lib/php/extensions/no-debug-non-zts-20100525/xdebug.so - -d - memory_limit=-1 - vendor/bin/phpunit If the value is null , the default entrypoint from the image is used. If the value is [] (empty list) or '' (empty string), the default entrypoint declared by the image is ignored, i.e. overridden to be empty.","title":"entrypoint"},{"location":"docker/#env_file","text":"env_file adds environment variables to the container based on the file content. env_file: .env env_file can also be a list. The files in the list are processed from the top down. For the same variable specified in two env files, the value from the last file in the list stands. env_file: - ./a.env - ./b.env List elements can also be declared as a mapping, which then lets you set an additional attribute required . This defaults to true . When required is set to false and the .env file is missing, Compose silently ignores the entry. env_file: - path: ./default.env required: true ## default - path: ./override.env required: false Relative path are resolved from the Compose file's parent folder. As absolute paths prevent the Compose file from being portable, Compose warns you when such a path is used to set env_file . Environment variables declared in the environment section override these values. This holds true even if those values are empty or undefined.","title":"env_file"},{"location":"docker/#env_file-format","text":"Each line in an .env file must be in VAR[=[VAL]] format. The following syntax rules apply: Lines beginning with # are processed as comments and ignored. Blank lines are ignored. Unquoted and double-quoted ( \" ) values have Interpolation applied. Each line represents a key-value pair. Values can optionally be quoted. VAR=VAL -> VAL VAR=\"VAL\" -> VAL VAR='VAL' -> VAL Inline comments for unquoted values must be preceded with a space. VAR=VAL ## comment -> VAL VAR=VAL## not a comment -> VAL## not a comment Inline comments for quoted values must follow the closing quote. VAR=\"VAL ## not a comment\" -> VAL ## not a comment VAR=\"VAL\" ## comment -> VAL Single-quoted ( ' ) values are used literally. VAR='$OTHER' -> $OTHER VAR='${OTHER}' -> ${OTHER} Quotes can be escaped with \\ . VAR='Let\\'s go!' -> Let's go! VAR=\"{\\\"hello\\\": \\\"json\\\"}\" -> {\"hello\": \"json\"} Common shell escape sequences including \\n , \\r , \\t , and \\\\ are supported in double-quoted values. VAR=\"some\\tvalue\" -> some value VAR='some\\tvalue' -> some\\tvalue VAR=some\\tvalue -> some\\tvalue VAL may be omitted, in such cases the variable value is an empty string. =VAL may be omitted, in such cases the variable is unset. ## Set Rails/Rack environment RACK_ENV=development VAR=\"quoted\"","title":"Env_file format"},{"location":"docker/#environment","text":"environment defines environment variables set in the container. environment can use either an array or a map. Any boolean values; true, false, yes, no, should be enclosed in quotes to ensure they are not converted to True or False by the YAML parser. Environment variables can be declared by a single key (no value to equals sign). In this case Compose relies on you to resolve the value. If the value is not resolved, the variable is unset and is removed from the service container environment. Map syntax: environment: RACK_ENV: development SHOW: \"true\" USER_INPUT: Array syntax: environment: - RACK_ENV=development - SHOW=true - USER_INPUT When both env_file and environment are set for a service, values set by environment have precedence.","title":"environment"},{"location":"docker/#expose","text":"expose defines the (incoming) port or a range of ports that Compose exposes from the container. These ports must be accessible to linked services and should not be published to the host machine. Only the internal container ports can be specified. Syntax is /[] or /[] for a port range. When not explicitly set, tcp protocol is used. expose: - \"3000\" - \"8000\" - \"8080-8085/tcp Note If the Dockerfile for the image already exposes ports, it is visible to other containers on the network even if expose is not set in your Compose file.","title":"expose"},{"location":"docker/#extends","text":"extends lets you share common configurations among different files, or even different projects entirely. With extends you can define a common set of service options in one place and refer to it from anywhere. You can refer to another Compose file and select a service you want to also use in your own application, with the ability to override some attributes for your own needs. You can use extends on any service together with other configuration keys. The extends value must be a mapping defined with a required service and an optional file key. extends: file: common.yml service: webapp service : Defines the name of the service being referenced as a base, for example web or database . file : The location of a Compose configuration file defining that service.","title":"extends"},{"location":"docker/#restrictions","text":"The following restrictions apply to the service being referenced: Services that have dependencies on other services cannot be used as a base. Therefore, any key that introduces a dependency on another service is incompatible with extends . The non-exhaustive list of such keys is: links , volumes_from , container mode (in ipc , pid , network_mode and net ), service mode (in ipc , pid and network_mode ), depends_on . Services cannot have circular references with extends . Compose returns an error in all of these cases.","title":"Restrictions"},{"location":"docker/#finding-referenced-service","text":"file value can be: Not present. This indicates that another service within the same Compose file is being referenced. File path, which can be either: Relative path. This path is considered as relative to the location of the main Compose file. Absolute path. A service denoted by service must be present in the identified referenced Compose file. Compose returns an error if: The service denoted by service is not found. The Compose file denoted by file is not found.","title":"Finding referenced service"},{"location":"docker/#merging-service-definitions","text":"Two service definitions, the main one in the current Compose file and the referenced one specified by extends , are merged in the following way: Mappings: Keys in mappings of the main service definition override keys in mappings of the referenced service definition. Keys that aren't overridden are included as is. Sequences: Items are combined together into a new sequence. The order of elements is preserved with the referenced items coming first and main items after. Scalars: Keys in the main service definition take precedence over keys in the referenced one.","title":"Merging service definitions"},{"location":"docker/#mappings","text":"The following keys should be treated as mappings: annotations , build.args , build.labels , build.extra_hosts , deploy.labels , deploy.update_config , deploy.rollback_config , deploy.restart_policy , deploy.resources.limits , environment , healthcheck , labels , logging.options , sysctls , storage_opt , extra_hosts , ulimits . One exception that applies to healthcheck is that the main mapping cannot specify disable: true unless the referenced mapping also specifies disable: true . Compose returns an error in this case. For example, the input below: services: common: image: busybox environment: TZ: utc PORT: 80 cli: extends: service: common environment: PORT: 8080 Produces the following configuration for the cli service. The same output is produced if array syntax is used. environment: PORT: 8080 TZ: utc image: busybox Items under blkio_config.device_read_bps , blkio_config.device_read_iops , blkio_config.device_write_bps , blkio_config.device_write_iops , devices and volumes are also treated as mappings where key is the target path inside the container. For example, the input below: services: common: image: busybox volumes: - common-volume:/var/lib/backup/data:rw cli: extends: service: common volumes: - cli-volume:/var/lib/backup/data:ro Produces the following configuration for the cli service. Note that the mounted path now points to the new volume name and ro flag was applied. image: busybox volumes: - cli-volume:/var/lib/backup/data:ro If the referenced service definition contains extends mapping, the items under it are simply copied into the new merged definition. The merging process is then kicked off again until no extends keys are remaining. For example, the input below: services: base: image: busybox user: root common: image: busybox extends: service: base cli: extends: service: common Produces the following configuration for the cli service. Here, cli services gets user key from common service, which in turn gets this key from base service. image: busybox user: root","title":"Mappings"},{"location":"docker/#sequences","text":"The following keys should be treated as sequences: cap_add , cap_drop , configs , deploy.placement.constraints , deploy.placement.preferences , deploy.reservations.generic_resources , device_cgroup_rules , expose , external_links , ports , secrets , security_opt . Any duplicates resulting from the merge are removed so that the sequence only contains unique elements. For example, the input below: services: common: image: busybox security_opt: - label:role:ROLE cli: extends: service: common security_opt: - label:user:USER Produces the following configuration for the cli service. image: busybox security_opt: - label:role:ROLE - label:user:USER In case list syntax is used, the following keys should also be treated as sequences: dns , dns_search , env_file , tmpfs . Unlike sequence fields mentioned above, duplicates resulting from the merge are not removed.","title":"Sequences"},{"location":"docker/#scalars","text":"Any other allowed keys in the service definition should be treated as scalars.","title":"Scalars"},{"location":"docker/#annotations","text":"annotations defines annotations for the container. annotations can use either an array or a map. annotations: com.example.foo: bar annotations: - com.example.foo=bar","title":"annotations"},{"location":"docker/#external_links","text":"external_links link service containers to services managed outside of your Compose application. external_links define the name of an existing service to retrieve using the platform lookup mechanism. An alias of the form SERVICE:ALIAS can be specified. external_links: - redis - database:mysql - database:postgresql","title":"external_links"},{"location":"docker/#extra_hosts","text":"extra_hosts adds hostname mappings to the container network interface configuration ( /etc/hosts for Linux).","title":"extra_hosts"},{"location":"docker/#short-syntax_2","text":"Short syntax uses plain strings in a list. Values must set hostname and IP address for additional hosts in the form of HOSTNAME=IP . extra_hosts: - \"somehost=162.242.195.82\" - \"otherhost=50.31.209.229\" - \"myhostv6=::1\" IPv6 addresses can be enclosed in square brackets, for example: extra_hosts: - \"myhostv6=[::1]\" The separator = is preferred, but : can also be used. For example: extra_hosts: - \"somehost:162.242.195.82\" - \"myhostv6:::1\"","title":"Short syntax"},{"location":"docker/#long-syntax_2","text":"Alternatively, extra_hosts can be set as a mapping between hostname(s) and IP(s) extra_hosts: somehost: \"162.242.195.82\" otherhost: \"50.31.209.229\" myhostv6: \"::1\" Compose creates a matching entry with the IP address and hostname in the container's network configuration, which means for Linux /etc/hosts get extra lines: 162.242.195.82 somehost 50.31.209.229 otherhost ::1 myhostv6","title":"Long syntax"},{"location":"docker/#group_add","text":"group_add specifies additional groups, by name or number, which the user inside the container must be a member of. An example of where this is useful is when multiple containers (running as different users) need to all read or write the same file on a shared volume. That file can be owned by a group shared by all the containers, and specified in group_add . services: myservice: image: alpine group_add: - mail Running id inside the created container must show that the user belongs to the mail group, which would not have been the case if group_add were not declared.","title":"group_add"},{"location":"docker/#healthcheck","text":"healthcheck declares a check that's run to determine whether or not the service containers are \"healthy\". It works in the same way, and has the same default values, as the HEALTHCHECK Dockerfile instruction set by the service's Docker image. Your Compose file can override the values set in the Dockerfile. healthcheck: test: [\"CMD\", \"curl\", \"-f\", \"http://localhost\"] interval: 1m30s timeout: 10s retries: 3 start_period: 40s start_interval: 5s interval , timeout , start_period , and start_interval are specified as durations . test defines the command Compose runs to check container health. It can be either a string or a list. If it's a list, the first item must be either NONE , CMD or CMD-SHELL . If it's a string, it's equivalent to specifying CMD-SHELL followed by that string. ## Hit the local web app test: [\"CMD\", \"curl\", \"-f\", \"http://localhost\"] Using CMD-SHELL runs the command configured as a string using the container's default shell ( /bin/sh for Linux). Both forms below are equivalent: test: [\"CMD-SHELL\", \"curl -f http://localhost || exit 1\"] test: curl -f https://localhost || exit 1 NONE disables the healthcheck, and is mostly useful to disable the Healthcheck Dockerfile instruction set by the service's Docker image. Alternatively, the healthcheck set by the image can be disabled by setting disable: true : healthcheck: disable: true","title":"healthcheck"},{"location":"docker/#hostname","text":"hostname declares a custom host name to use for the service container. It must be a valid RFC 1123 hostname.","title":"hostname"},{"location":"docker/#image","text":"image specifies the image to start the container from. image must follow the Open Container Specification addressable image format , as [/][/][:|@] . image: redis image: redis:5 image: redis@sha256:0ed5d5928d4737458944eb604cc8509e245c3e19d02ad83935398bc4b991aac7 image: library/redis image: docker.io/library/redis image: my_private.registry:5000/redis If the image does not exist on the platform, Compose attempts to pull it based on the pull_policy . If you are also using the Compose Build Specification , there are alternative options for controlling the precedence of pull over building the image from source, however pulling the image is the default behavior. image may be omitted from a Compose file as long as a build section is declared. If you are not using the Compose Build Specification, Compose won't work if image is missing from the Compose file.","title":"image"},{"location":"docker/#init","text":"init runs an init process (PID 1) inside the container that forwards signals and reaps processes. Set this option to true to enable this feature for the service. services: web: image: alpine:latest init: true The init binary that is used is platform specific.","title":"init"},{"location":"docker/#ipc","text":"ipc configures the IPC isolation mode set by the service container. Available values are platform specific, but Compose defines specific values which must be implemented as described if supported: shareable : Gives the container its own private IPC namespace, with a possibility to share it with other containers. service:{name} : Makes the container join another container's ( shareable ) IPC namespace. ipc: \"shareable\" ipc: \"service:[service name]\"","title":"ipc"},{"location":"docker/#uts","text":"uts configures the UTS namespace mode set for the service container. When unspecified it is the runtime's decision to assign a UTS namespace, if supported. Available values are: 'host' : Results in the container using the same UTS namespace as the host. uts: \"host\"","title":"uts"},{"location":"docker/#isolation","text":"isolation specifies a container\u2019s isolation technology. Supported values are platform specific.","title":"isolation"},{"location":"docker/#labels","text":"labels add metadata to containers. You can use either an array or a map. It's recommended that you use reverse-DNS notation to prevent your labels from conflicting with those used by other software. labels: com.example.description: \"Accounting webapp\" com.example.department: \"Finance\" com.example.label-with-empty-value: \"\" labels: - \"com.example.description=Accounting webapp\" - \"com.example.department=Finance\" - \"com.example.label-with-empty-value\" Compose creates containers with canonical labels: com.docker.compose.project set on all resources created by Compose to the user project name com.docker.compose.service set on service containers with service name as defined in the Compose file The com.docker.compose label prefix is reserved. Specifying labels with this prefix in the Compose file results in a runtime error.","title":"labels"},{"location":"docker/#links","text":"Note Availability of the links attribute is implementation specific. links defines a network link to containers in another service. Either specify both the service name and a link alias ( SERVICE:ALIAS ), or just the service name. web: links: - db - db:database - redis Containers for the linked service are reachable at a hostname identical to the alias, or the service name if no alias is specified. Links are not required to enable services to communicate. When no specific network configuration is set, any service is able to reach any other service at that service\u2019s name on the default network. If services do declare networks they are attached to, links does not override the network configuration and services not attached to a shared network are not be able to communicate. Compose doesn't warn you about a configuration mismatch. Links also express implicit dependency between services in the same way as depends_on , so they determine the order of service startup.","title":"links"},{"location":"docker/#logging","text":"logging defines the logging configuration for the service. logging: driver: syslog options: syslog-address: \"tcp://192.168.0.42:123\" The driver name specifies a logging driver for the service's containers. The default and available values are platform specific. Driver specific options can be set with options as key-value pairs.","title":"logging"},{"location":"docker/#network_mode","text":"network_mode sets a service container's network mode. Available values are platform specific, but Compose defines specific values which must be implemented as described if supported: none : Turns off all container networking. host : Gives the container raw access to the host's network interface. service:{name} : Gives the containers access to the specified service only. network_mode: \"host\" network_mode: \"none\" network_mode: \"service:[service name]\" When set, the networks attribute is not allowed and Compose rejects any Compose file containing both attributes.","title":"network_mode"},{"location":"docker/#networks","text":"networks defines the networks that service containers are attached to, referencing entries under the top-level networks key . services: some-service: networks: - some-network - other-network","title":"networks"},{"location":"docker/#aliases","text":"aliases declares alternative hostnames for the service on the network. Other containers on the same network can use either the service name or an alias to connect to one of the service's containers. Since aliases are network-scoped, the same service can have different aliases on different networks. Note A network-wide alias can be shared by multiple containers, and even by multiple services. If it is, then exactly which container the name resolves to is not guaranteed. services: some-service: networks: some-network: aliases: - alias1 - alias3 other-network: aliases: - alias2 In the following example, service frontend is able to reach the backend service at the hostname backend or database on the back-tier network. The service monitoring is able to reach same backend service at backend or mysql on the admin network. services: frontend: image: example/webapp networks: - front-tier - back-tier monitoring: image: example/monitoring networks: - admin backend: image: example/backend networks: back-tier: aliases: - database admin: aliases: - mysql networks: front-tier: back-tier: admin:","title":"aliases"},{"location":"docker/#ipv4_address-ipv6_address","text":"Specify a static IP address for a service container when joining the network. The corresponding network configuration in the top-level networks section must have an ipam attribute with subnet configurations covering each static address. services: frontend: image: example/webapp networks: front-tier: ipv4_address: 172.16.238.10 ipv6_address: 2001:3984:3989::10 networks: front-tier: ipam: driver: default config: - subnet: \"172.16.238.0/24\" - subnet: \"2001:3984:3989::/64\"","title":"ipv4_address, ipv6_address"},{"location":"docker/#link_local_ips","text":"link_local_ips specifies a list of link-local IPs. Link-local IPs are special IPs which belong to a well known subnet and are purely managed by the operator, usually dependent on the architecture where they are deployed. Implementation is platform specific. Example: services: app: image: busybox command: top networks: app_net: link_local_ips: - 57.123.22.11 - 57.123.22.13 networks: app_net: driver: bridge","title":"link_local_ips"},{"location":"docker/#mac_address","text":"mac_address sets the MAC address used by the service container when connecting to this particular network.","title":"mac_address"},{"location":"docker/#priority","text":"priority indicates in which order Compose connects the service\u2019s containers to its networks. If unspecified, the default value is 0. In the following example, the app service connects to app_net_1 first as it has the highest priority. It then connects to app_net_3 , then app_net_2 , which uses the default priority value of 0. services: app: image: busybox command: top networks: app_net_1: priority: 1000 app_net_2: app_net_3: priority: 100 networks: app_net_1: app_net_2: app_net_3:","title":"priority"},{"location":"docker/#mac_address_1","text":"mac_address sets a MAC address for the service container. Note Container runtimes might reject this value (ie. Docker Engine >= v25.0). In that case, you should use networks.mac_address instead.","title":"mac_address"},{"location":"docker/#mem_limit","text":"DEPRECATED: use deploy.limits.memory","title":"mem_limit"},{"location":"docker/#mem_reservation","text":"DEPRECATED: use deploy.reservations.memory","title":"mem_reservation"},{"location":"docker/#mem_swappiness","text":"mem_swappiness defines as a percentage, a value between 0 and 100, for the host kernel to swap out anonymous memory pages used by a container. 0 : Turns off anonymous page swapping. 100 : Sets all anonymous pages as swappable. The default value is platform specific.","title":"mem_swappiness"},{"location":"docker/#memswap_limit","text":"memswap_limit defines the amount of memory the container is allowed to swap to disk. This is a modifier attribute that only has meaning if memory is also set. Using swap lets the container write excess memory requirements to disk when the container has exhausted all the memory that is available to it. There is a performance penalty for applications that swap memory to disk often. If memswap_limit is set to a positive integer, then both memory and memswap_limit must be set. memswap_limit represents the total amount of memory and swap that can be used, and memory controls the amount used by non-swap memory. So if memory =\"300m\" and memswap_limit =\"1g\", the container can use 300m of memory and 700m (1g - 300m) swap. If memswap_limit is set to 0, the setting is ignored, and the value is treated as unset. If memswap_limit is set to the same value as memory , and memory is set to a positive integer, the container does not have access to swap. If memswap_limit is unset, and memory is set, the container can use as much swap as the memory setting, if the host container has swap memory configured. For instance, if memory =\"300m\" and memswap_limit is not set, the container can use 600m in total of memory and swap. If memswap_limit is explicitly set to -1, the container is allowed to use unlimited swap, up to the amount available on the host system.","title":"memswap_limit"},{"location":"docker/#oom_kill_disable","text":"If oom_kill_disable is set, Compose configures the platform so it won't kill the container in case of memory starvation.","title":"oom_kill_disable"},{"location":"docker/#oom_score_adj","text":"oom_score_adj tunes the preference for containers to be killed by platform in case of memory starvation. Value must be within [-1000,1000] range.","title":"oom_score_adj"},{"location":"docker/#pid","text":"pid sets the PID mode for container created by Compose. Supported values are platform specific.","title":"pid"},{"location":"docker/#pids_limit","text":"DEPRECATED: use deploy.resources.limits.pids pids_limit tunes a container\u2019s PIDs limit. Set to -1 for unlimited PIDs. pids_limit: 10","title":"pids_limit"},{"location":"docker/#platform","text":"platform defines the target platform the containers for the service run on. It uses the os[/arch[/variant]] syntax. The values of os , arch , and variant must conform to the convention used by the OCI Image Spec . Compose uses this attribute to determine which version of the image is pulled and/or on which platform the service\u2019s build is performed. platform: darwin platform: windows/amd64 platform: linux/arm64/v8","title":"platform"},{"location":"docker/#ports","text":"Exposes container ports. Note Port mapping must not be used with network_mode: host otherwise a runtime error occurs.","title":"ports"},{"location":"docker/#short-syntax_3","text":"The short syntax is a colon-separated string to set the host IP, host port, and container port in the form: [HOST:]CONTAINER[/PROTOCOL] where: HOST is [IP:](port | range) CONTAINER is port | range PROTOCOL to restrict port to specified protocol. tcp and udp values are defined by the Specification, Compose offers support for platform-specific protocol names. If host IP is not set, it binds to all network interfaces. Ports can be either a single value or a range. Host and container must use equivalent ranges. Either specify both ports ( HOST:CONTAINER ), or just the container port. In the latter case, the container runtime automatically allocates any unassigned port of the host. HOST:CONTAINER should always be specified as a (quoted) string, to avoid conflicts with yaml base-60 float . Examples: ports: - \"3000\" - \"3000-3005\" - \"8000:8000\" - \"9090-9091:8080-8081\" - \"49100:22\" - \"8000-9000:80\" - \"127.0.0.1:8001:8001\" - \"127.0.0.1:5000-5010:5000-5010\" - \"6060:6060/udp\" Note If Host IP mapping is not supported by a container engine, Compose rejects the Compose file and ignores the specified host IP.","title":"Short syntax"},{"location":"docker/#long-syntax_3","text":"The long form syntax allows the configuration of additional fields that can't be expressed in the short form. target : The container port published : The publicly exposed port. It is defined as a string and can be set as a range using syntax start-end . It means the actual port is assigned a remaining available port, within the set range. host_ip : The Host IP mapping, unspecified means all network interfaces ( 0.0.0.0 ). protocol : The port protocol ( tcp or udp ). Defaults to tcp . mode : host : For publishing a host port on each node, or ingress for a port to be load balanced. Defaults to ingress . name : A human-readable name for the port, used to document it's usage within the service ports: - name: http target: 80 host_ip: 127.0.0.1 published: \"8080\" protocol: tcp mode: host - name: https target: 443 host_ip: 127.0.0.1 published: \"8083-9000\" protocol: tcp mode: host","title":"Long syntax"},{"location":"docker/#privileged","text":"privileged configures the service container to run with elevated privileges. Support and actual impacts are platform specific.","title":"privileged"},{"location":"docker/#profiles","text":"profiles defines a list of named profiles for the service to be enabled under. If unassigned, the service is always started but if assigned, it is only started if the profile is activated. If present, profiles follow the regex format of [a-zA-Z0-9][a-zA-Z0-9_.-]+ . services: frontend: image: frontend profiles: [\"frontend\"] phpmyadmin: image: phpmyadmin depends_on: - db profiles: - debug","title":"profiles"},{"location":"docker/#pull_policy","text":"pull_policy defines the decisions Compose makes when it starts to pull images. Possible values are: always : Compose always pulls the image from the registry. never : Compose doesn't pull the image from a registry and relies on the platform cached image. If there is no cached image, a failure is reported. missing : Compose pulls the image only if it's not available in the platform cache. This is the default option if you are not also using the Compose Build Specification . if_not_present is considered an alias for this value for backward compatibility. build : Compose builds the image. Compose rebuilds the image if it's already present.","title":"pull_policy"},{"location":"docker/#read_only","text":"read_only configures the service container to be created with a read-only filesystem.","title":"read_only"},{"location":"docker/#restart","text":"restart defines the policy that the platform applies on container termination. no : The default restart policy. It does not restart the container under any circumstances. always : The policy always restarts the container until its removal. on-failure : The policy restarts the container if the exit code indicates an error. unless-stopped : The policy restarts the container irrespective of the exit code but stops restarting when the service is stopped or removed. restart: \"no\" restart: always restart: on-failure restart: unless-stopped","title":"restart"},{"location":"docker/#runtime","text":"runtime specifies which runtime to use for the service\u2019s containers. The value of runtime is specific to the implementation. For example, runtime can be the name of an implementation of OCI Runtime Spec , such as \"runc\". web: image: busybox:latest command: true runtime: runc","title":"runtime"},{"location":"docker/#scale","text":"scale specifies the default number of containers to deploy for this service. When both are set, scale must be consistent with the replicas attribute in the Deploy Specification .","title":"scale"},{"location":"docker/#secrets","text":"secrets grants access to sensitive data defined by secrets on a per-service basis. Two different syntax variants are supported; the short syntax and the long syntax. Compose reports an error if the secret doesn't exist on the platform or isn't defined in the secrets section of the Compose file. Services can be granted access to multiple secrets. Long and short syntax for secrets may be used in the same Compose file. Defining a secret in the top-level secrets must not imply granting any service access to it. Such grant must be explicit within service specification as secrets service element.","title":"secrets"},{"location":"docker/#short-syntax_4","text":"The short syntax variant only specifies the secret name. This grants the container access to the secret and mounts it as read-only to /run/secrets/ within the container. The source name and destination mountpoint are both set to the secret name. The following example uses the short syntax to grant the frontend service access to the server-certificate secret. The value of server-certificate is set to the contents of the file ./server.cert . services: frontend: image: example/webapp secrets: - server-certificate secrets: server-certificate: file: ./server.cert","title":"Short syntax"},{"location":"docker/#long-syntax_4","text":"The long syntax provides more granularity in how the secret is created within the service's containers. source : The name of the secret as it exists on the platform. target : The name of the file to be mounted in /run/secrets/ in the service's task container, or absolute path of the file if an alternate location is required. Defaults to source if not specified. uid and gid : The numeric UID or GID that owns the file within /run/secrets/ in the service's task containers. Default value is USER running container. mode : The permissions for the file to be mounted in /run/secrets/ in the service's task containers, in octal notation. The default value is world-readable permissions (mode 0444 ). The writable bit must be ignored if set. The executable bit may be set. Note that the uid , gid , and mode attributes are implementation specific. The following example sets the name of the server-certificate secret file to server.crt within the container, sets the mode to 0440 (group-readable), and sets the user and group to 103 . The value of server-certificate secret is provided by the platform through a lookup and the secret's lifecycle is not directly managed by Compose. services: frontend: image: example/webapp secrets: - source: server-certificate target: server.cert uid: \"103\" gid: \"103\" mode: 0440 secrets: server-certificate: external: true","title":"Long syntax"},{"location":"docker/#security_opt","text":"security_opt overrides the default labeling scheme for each container. security_opt: - label:user:USER - label:role:ROLE For further default labeling schemes you can override, see Security configuration .","title":"security_opt"},{"location":"docker/#shm_size","text":"shm_size configures the size of the shared memory ( /dev/shm partition on Linux) allowed by the service container. It's specified as a byte value .","title":"shm_size"},{"location":"docker/#stdin_open","text":"stdin_open configures a service containers to run with an allocated stdin.","title":"stdin_open"},{"location":"docker/#stop_grace_period","text":"stop_grace_period specifies how long Compose must wait when attempting to stop a container if it doesn't handle SIGTERM (or whichever stop signal has been specified with stop_signal ), before sending SIGKILL. It's specified as a duration . stop_grace_period: 1s stop_grace_period: 1m30s Default value is 10 seconds for the container to exit before sending SIGKILL.","title":"stop_grace_period"},{"location":"docker/#stop_signal","text":"stop_signal defines the signal that Compose uses to stop the service containers. If unset containers are stopped by Compose by sending SIGTERM . stop_signal: SIGUSR1","title":"stop_signal"},{"location":"docker/#storage_opt","text":"storage_opt defines storage driver options for a service. storage_opt: size: '1G'","title":"storage_opt"},{"location":"docker/#sysctls","text":"sysctls defines kernel parameters to set in the container. sysctls can use either an array or a map. sysctls: net.core.somaxconn: 1024 net.ipv4.tcp_syncookies: 0 sysctls: - net.core.somaxconn=1024 - net.ipv4.tcp_syncookies=0 You can only use sysctls that are namespaced in the kernel. Docker does not support changing sysctls inside a container that also modify the host system. For an overview of supported sysctls, refer to configure namespaced kernel parameters (sysctls) at runtime .","title":"sysctls"},{"location":"docker/#tmpfs","text":"tmpfs mounts a temporary file system inside the container. It can be a single value or a list. tmpfs: /run tmpfs: - /run - /tmp","title":"tmpfs"},{"location":"docker/#tty","text":"tty configures service container to run with a TTY.","title":"tty"},{"location":"docker/#ulimits","text":"ulimits overrides the default ulimits for a container. It's specified either as an integer for a single limit or as mapping for soft/hard limits. ulimits: nproc: 65535 nofile: soft: 20000 hard: 40000","title":"ulimits"},{"location":"docker/#user","text":"user overrides the user used to run the container process. The default is set by the image (i.e. Dockerfile USER ). If it's not set, then root .","title":"user"},{"location":"docker/#userns_mode","text":"userns_mode sets the user namespace for the service. Supported values are platform specific and may depend on platform configuration. userns_mode: \"host\"","title":"userns_mode"},{"location":"docker/#volumes","text":"volumes define mount host paths or named volumes that are accessible by service containers. You can use volumes to define multiple types of mounts; volume , bind , tmpfs , or npipe . If the mount is a host path and is only used by a single service, it can be declared as part of the service definition. To reuse a volume across multiple services, a named volume must be declared in the top-level volumes key . The following example shows a named volume ( db-data ) being used by the backend service, and a bind mount defined for a single service. services: backend: image: example/backend volumes: - type: volume source: db-data target: /data volume: nocopy: true - type: bind source: /var/run/postgres/postgres.sock target: /var/run/postgres/postgres.sock volumes: db-data:","title":"volumes"},{"location":"docker/#short-syntax_5","text":"The short syntax uses a single string with colon-separated values to specify a volume mount ( VOLUME:CONTAINER_PATH ), or an access mode ( VOLUME:CONTAINER_PATH:ACCESS_MODE ). VOLUME : Can be either a host path on the platform hosting containers (bind mount) or a volume name. CONTAINER_PATH : The path in the container where the volume is mounted. ACCESS_MODE : A comma-separated , list of options: rw : Read and write access. This is the default if none is specified. ro : Read-only access. z : SELinux option indicating that the bind mount host content is shared among multiple containers. Z : SELinux option indicating that the bind mount host content is private and unshared for other containers. Note The SELinux re-labeling bind mount option is ignored on platforms without SELinux. Note Relative host paths are only supported by Compose that deploy to a local container runtime. This is because the relative path is resolved from the Compose file\u2019s parent directory which is only applicable in the local case. When Compose deploys to a non-local platform it rejects Compose files which use relative host paths with an error. To avoid ambiguities with named volumes, relative paths should always begin with . or .. .","title":"Short syntax"},{"location":"docker/#long-syntax_5","text":"The long form syntax allows the configuration of additional fields that can't be expressed in the short form. type : The mount type. Either volume , bind , tmpfs , npipe , or cluster source : The source of the mount, a path on the host for a bind mount, or the name of a volume defined in the top-level volumes key . Not applicable for a tmpfs mount. target : The path in the container where the volume is mounted. read_only : Flag to set the volume as read-only. bind : Used to configure additional bind options: propagation : The propagation mode used for the bind. create_host_path : Creates a directory at the source path on host if there is nothing present. Compose does nothing if there is something present at the path. This is automatically implied by short syntax for backward compatibility with docker-compose legacy. selinux : The SELinux re-labeling option z (shared) or Z (private) volume : Configures additional volume options: nocopy : Flag to disable copying of data from a container when a volume is created. tmpfs : Configures additional tmpfs options: size : The size for the tmpfs mount in bytes (either numeric or as bytes unit). mode : The file mode for the tmpfs mount as Unix permission bits as an octal number. consistency : The consistency requirements of the mount. Available values are platform specific.","title":"Long syntax"},{"location":"docker/#volumes_from","text":"volumes_from mounts all of the volumes from another service or container. You can optionally specify read-only access ro or read-write rw . If no access level is specified, then read-write access is used. You can also mount volumes from a container that is not managed by Compose by using the container: prefix. volumes_from: - service_name - service_name:ro - container:container_name - container:container_name:rw","title":"volumes_from"},{"location":"docker/#working_dir","text":"working_dir overrides the container's working directory which is specified by the image, for example Dockerfile's WORKDIR .","title":"working_dir"},{"location":"docker/#networks-top-level-element","text":"Networks are the layer that allow services to communicate with each other. The top-level networks element lets you configure named networks that can be reused across multiple services. To use a network across multiple services, you must explicitly grant each service access by using the networks attribute within the services top-level element. The networks top-level element has additional syntax that provides more granular control.","title":"Networks top-level element"},{"location":"docker/#examples","text":"","title":"Examples"},{"location":"docker/#basic-example","text":"In the following example, at runtime, networks front-tier and back-tier are created and the frontend service is connected to front-tier and back-tier networks. services: frontend: image: example/webapp networks: - front-tier - back-tier networks: front-tier: back-tier:","title":"Basic example"},{"location":"docker/#advanced-example","text":"services: proxy: build: ./proxy networks: - frontend app: build: ./app networks: - frontend - backend db: image: postgres networks: - backend networks: frontend: ## Use a custom driver driver: custom-driver-1 backend: ## Use a custom driver which takes special options driver: custom-driver-2 driver_opts: foo: \"1\" bar: \"2\" The advanced example shows a Compose file which defines two custom networks. The proxy service is isolated from the db service, because they do not share a network in common. Only app can talk to both.","title":"Advanced example"},{"location":"docker/#attributes","text":"","title":"Attributes"},{"location":"docker/#driver","text":"driver specifies which driver should be used for this network. Compose returns an error if the driver is not available on the platform. networks: db-data: driver: overlay Default and available values are platform specific. Compose supports the following drivers: none and host host : Use the host's networking stack. none : Turn off networking.","title":"driver"},{"location":"docker/#host-or-none","text":"The syntax for using built-in networks such as host and none is different, as such networks implicitly exist outside the scope of Compose. To use them, you must define an external network with the name host or none and an alias that Compose can use ( hostnet and nonet in the following example), then grant the service access to that network using its alias. services: web: networks: hostnet: {} networks: hostnet: external: true name: host services: web: ... networks: nonet: {} networks: nonet: external: true name: none","title":"host or none"},{"location":"docker/#driver_opts","text":"driver_opts specifies a list of options as key-value pairs to pass to the driver. These options are driver-dependent. Consult the driver's documentation for more information. networks: db-data: driver_opts: foo: \"bar\" baz: 1","title":"driver_opts"},{"location":"docker/#attachable","text":"If attachable is set to true , then standalone containers should be able to attach to this network, in addition to services. If a standalone container attaches to the network, it can communicate with services and other standalone containers that are also attached to the network. networks: mynet1: driver: overlay attachable: true","title":"attachable"},{"location":"docker/#enable_ipv6","text":"enable_ipv6 enables IPv6 networking. For an example, see step four of Create an IPv6 network .","title":"enable_ipv6"},{"location":"docker/#external","text":"If set to true : - external specifies that this network\u2019s lifecycle is maintained outside of that of the application. Compose doesn't attempt to create these networks, and returns an error if one doesn't exist. - All other attributes apart from name are irrelevant. If Compose detects any other attribute, it rejects the Compose file as invalid. In the example below, proxy is the gateway to the outside world. Instead of attempting to create a network, Compose queries the platform for an existing network simply called outside and connects the proxy service's containers to it. services: proxy: image: example/proxy networks: - outside - default app: image: example/app networks: - default networks: outside: external: true","title":"external"},{"location":"docker/#ipam","text":"ipam specifies a custom IPAM configuration. This is an object with several properties, each of which is optional: driver : Custom IPAM driver, instead of the default. config : A list with zero or more configuration elements, each containing a: subnet : Subnet in CIDR format that represents a network segment ip_range : Range of IPs from which to allocate container IPs gateway : IPv4 or IPv6 gateway for the master subnet aux_addresses : Auxiliary IPv4 or IPv6 addresses used by Network driver, as a mapping from hostname to IP options : Driver-specific options as a key-value mapping. networks: mynet1: ipam: driver: default config: - subnet: 172.28.0.0/16 ip_range: 172.28.5.0/24 gateway: 172.28.5.254 aux_addresses: host1: 172.28.1.5 host2: 172.28.1.6 host3: 172.28.1.7 options: foo: bar baz: \"0\"","title":"ipam"},{"location":"docker/#internal","text":"By default, Compose provides external connectivity to networks. internal , when set to true , allows you to create an externally isolated network.","title":"internal"},{"location":"docker/#labels_1","text":"Add metadata to containers using labels . You can use either an array or a dictionary. It is recommended that you use reverse-DNS notation to prevent labels from conflicting with those used by other software. networks: mynet1: labels: com.example.description: \"Financial transaction network\" com.example.department: \"Finance\" com.example.label-with-empty-value: \"\" networks: mynet1: labels: - \"com.example.description=Financial transaction network\" - \"com.example.department=Finance\" - \"com.example.label-with-empty-value\" Compose sets com.docker.compose.project and com.docker.compose.network labels.","title":"labels"},{"location":"docker/#name","text":"name sets a custom name for the network. The name field can be used to reference networks which contain special characters. The name is used as is and is not scoped with the project name. networks: network1: name: my-app-net It can also be used in conjunction with the external property to define the platform network that Compose should retrieve, typically by using a parameter so the Compose file doesn't need to hard-code runtime specific values: networks: network1: external: true name: \"${NETWORK_ID}\"","title":"name"},{"location":"docker/#volumes-top-level-element","text":"Volumes are persistent data stores implemented by the container engine. Compose offers a neutral way for services to mount volumes, and configuration parameters to allocate them to infrastructure. The top-level volumes declaration lets you configure named volumes that can be reused across multiple services. To use a volume across multiple services, you must explicitly grant each service access by using the volumes attribute within the services top-level element. The volumes attribute has additional syntax that provides more granular control.","title":"Volumes top-level element"},{"location":"docker/#example","text":"The following example shows a two-service setup where a database's data directory is shared with another service as a volume, named db-data , so that it can be periodically backed up. services: backend: image: example/database volumes: - db-data:/etc/data backup: image: backup-service volumes: - db-data:/var/lib/backup/data volumes: db-data: The db-data volume is mounted at the /var/lib/backup/data and /etc/data container paths for backup and backend respectively. Running docker compose up creates the volume if it doesn't already exist. Otherwise, the existing volume is used and is recreated if it's manually deleted outside of Compose.","title":"Example"},{"location":"docker/#attributes_1","text":"An entry under the top-level volumes section can be empty, in which case it uses the container engine's default configuration for creating a volume. Optionally, you can configure it with the following keys:","title":"Attributes"},{"location":"docker/#driver_1","text":"Specifies which volume driver should be used. Default and available values are platform specific. If the driver is not available, Compose returns an error and doesn't deploy the application. volumes: db-data: driver: foobar","title":"driver"},{"location":"docker/#driver_opts_1","text":"driver_opts specifies a list of options as key-value pairs to pass to the driver for this volume. The options are driver-dependent. volumes: example: driver_opts: type: \"nfs\" o: \"addr=10.40.0.199,nolock,soft,rw\" device: \":/docker/example\"","title":"driver_opts"},{"location":"docker/#external_1","text":"If set to true : - external specifies that this volume already exists on the platform and its lifecycle is managed outside of that of the application. Compose doesn't then create the volume, and returns an error if the volume doesn't exist. - All other attributes apart from name are irrelevant. If Compose detects any other attribute, it rejects the Compose file as invalid. In the example below, instead of attempting to create a volume called {project_name}_db-data , Compose looks for an existing volume simply called db-data and mounts it into the backend service's containers. services: backend: image: example/database volumes: - db-data:/etc/data volumes: db-data: external: true","title":"external"},{"location":"docker/#labels_2","text":"labels are used to add metadata to volumes. You can use either an array or a dictionary. It's recommended that you use reverse-DNS notation to prevent your labels from conflicting with those used by other software. volumes: db-data: labels: com.example.description: \"Database volume\" com.example.department: \"IT/Ops\" com.example.label-with-empty-value: \"\" volumes: db-data: labels: - \"com.example.description=Database volume\" - \"com.example.department=IT/Ops\" - \"com.example.label-with-empty-value\" Compose sets com.docker.compose.project and com.docker.compose.volume labels.","title":"labels"},{"location":"docker/#name_1","text":"name sets a custom name for a volume. The name field can be used to reference volumes that contain special characters. The name is used as is and is not scoped with the stack name. volumes: db-data: name: \"my-app-data\" This makes it possible to make this lookup name a parameter of the Compose file, so that the model ID for the volume is hard-coded but the actual volume ID on the platform is set at runtime during deployment. For example, if DATABASE_VOLUME=my_volume_001 in your .env file: volumes: db-data: name: ${DATABASE_VOLUME} Running docker compose up uses the volume called my_volume_001 . It can also be used in conjunction with the external property. This means the name of the volume used to lookup the actual volume on the platform is set separately from the name used to refer to it within the Compose file: volumes: db-data: external: name: actual-name-of-volume","title":"name"},{"location":"docker/#configs-top-level-element","text":"Configs allow services to adapt their behaviour without the need to rebuild a Docker image. Services can only access configs when explicitly granted by a configs attribute within the services top-level element. As with volumes, configs are mounted as files into a service's container's filesystem. The location of the mount point within the container defaults to / in Linux containers and C:\\ in Windows containers. By default, the config: - Is owned by the user running the container command but can be overridden by service configuration. - Has world-readable permissions (mode 0444), unless the service is configured to override this. The top-level configs declaration defines or references configuration data that is granted to services in your Compose application. The source of the config is either file or external . file : The config is created with the contents of the file at the specified path. environment : The config content is created with the value of an environment variable. content : The content is created with the inlined value. external : If set to true, external specifies that this config has already been created. Compose does not attempt to create it, and if it does not exist, an error occurs. name : The name of the config object in the container engine to look up. This field can be used to reference configs that contain special characters. The name is used as is and will not be scoped with the project name.","title":"Configs top-level element"},{"location":"docker/#example-1","text":"_http_config is created when the application is deployed, by registering the content of the httpd.conf as the configuration data. configs: http_config: file: ./httpd.conf Alternatively, http_config can be declared as external. Compose looks up http_config to expose the configuration data to relevant services. configs: http_config: external: true","title":"Example 1"},{"location":"docker/#example-2","text":"_app_config is created when the application is deployed, by registering the inlined content as the configuration data. This comes with the benefits Compose will infer variables when creating the config, which allows to adjust content according to service configuration: configs: app_config: content: | debug=${DEBUG} spring.application.admin.enabled=${DEBUG} spring.application.name=${COMPOSE_PROJECT_NAME}","title":"Example 2"},{"location":"docker/#example-3","text":"External configs lookup can also use a distinct key by specifying a name . The following example modifies the previous one to look up a config using the parameter HTTP_CONFIG_KEY . The the actual lookup key will is set at deployment time by the interpolation of variables, but exposed to containers as hard-coded ID http_config . configs: http_config: external: true name: \"${HTTP_CONFIG_KEY}\" If external is set to true , all other attributes apart from name are irrelevant. If Compose detecs any other attribute, it rejects the Compose file as invalid.","title":"Example 3"},{"location":"docker/#secrets-top-level-element","text":"Secrets are a flavor of Configs focusing on sensitive data, with specific constraint for this usage. Services can only access secrets when explicitly granted by a secrets attribute within the services top-level element. The top-level secrets declaration defines or references sensitive data that is granted to the services in your Compose application. The source of the secret is either file or environment . file : The secret is created with the contents of the file at the specified path. environment : The secret is created with the value of an environment variable. external : If set to true, external specifies that this secret has already been created. Compose does not attempt to create it, and if it does not exist, an error occurs. name : The name of the secret object in Docker. This field can be used to reference secrets that contain special characters. The name is used as is and isn't scoped with the project name.","title":"Secrets top-level element"},{"location":"docker/#example-1_1","text":"server-certificate secret is created as _server-certificate when the application is deployed, by registering content of the server.cert as a platform secret. secrets: server-certificate: file: ./server.cert","title":"Example 1"},{"location":"docker/#example-2_1","text":"token secret is created as _token when the application is deployed, by registering the content of the OAUTH_TOKEN environment variable as a platform secret. secrets: token: environment: \"OAUTH_TOKEN\" Alternatively, server-certificate can be declared as external. Compose looks up the server-certificate secret to expose to relevant services. secrets: server-certificate: external: true","title":"Example 2"},{"location":"docker/#example-3_1","text":"External secrets lookup can also use a distinct key by specifying a name . The following example modifies the previous example to look up a secret using the name CERTIFICATE_KEY . The actual lookup key is set at deployment time by the interpolation of variables, but exposed to containers as hard-coded ID server-certificate . secrets: server-certificate: external: true name: \"${CERTIFICATE_KEY}\" If external is set to true , all other attributes apart from name are irrelevant. If Compose detects any other attribute, it rejects the Compose file as invalid. Your Compose file needs to explicitly grant access to the secrets to relevant services in your application.","title":"Example 3"},{"location":"docker/#fragments","text":"With Compose, you can use built-in YAML features to make your Compose file neater and more efficient. Anchors and aliases let you create re-usable blocks. This is useful if you start to find common configurations that span multiple services. Having re-usable blocks minimizes potential mistakes. Anchors are created using the & sign. The sign is followed by an alias name. You can use this alias with the * sign later to reference the value following the anchor. Make sure there is no space between the & and the * characters and the following alias name. You can use more than one anchor and alias in a single Compose file.","title":"Fragments"},{"location":"docker/#example-1_2","text":"volumes: db-data: &default-volume driver: default metrics: *default-volume In the example above, a default-volume anchor is created based on the db-data volume. It is later reused by the alias *default-volume to define the metrics volume. Anchor resolution takes place before variables interpolation , so variables can't be used to set anchors or aliases.","title":"Example 1"},{"location":"docker/#example-2_2","text":"services: first: image: my-image:latest environment: &env - CONFIG_KEY - EXAMPLE_KEY - DEMO_VAR second: image: another-image:latest environment: *env If you have an anchor that you want to use in more than one service, use it in conjunction with an extension to make your Compose file easier to maintain.","title":"Example 2"},{"location":"docker/#example-3_2","text":"You may want to partially override values. Compose follows the rule outlined by YAML merge type . In the following example, metrics volume specification uses alias to avoid repetition but overrides name attribute: services: backend: image: example/database volumes: - db-data - metrics volumes: db-data: &default-volume driver: default name: \"data\" metrics: <<: *default-volume name: \"metrics\"","title":"Example 3"},{"location":"docker/#example-4","text":"You can also extend the anchor to add additional values. services: first: image: my-image:latest environment: &env FOO: BAR ZOT: QUIX second: image: another-image:latest environment: <<: *env YET_ANOTHER: VARIABLE Note YAML merge only applies to mappings, and can't be used with sequences. In example above, the environment variables must be declared using the FOO: BAR mapping syntax, while the sequence syntax - FOO=BAR is only valid when no fragments are involved.","title":"Example 4"},{"location":"docker/#extension","text":"As with Fragments , Extensions can be used to make your Compose file more efficient and easier to maintain. Extensions can also be used with anchors and aliases . Use the prefix x- as a top-level element to modularize configurations that you want to reuse. Compose ignores any fields that start with x- , this is the sole exception where Compose silently ignores unrecognized fields. They also can be used within any structure in a Compose file where user-defined keys are not expected. Compose use those to enable experimental features, the same way browsers add support for custom CSS features","title":"Extension"},{"location":"docker/#example-1_3","text":"x-custom: foo: - bar - zot services: webapp: image: example/webapp x-foo: bar service: backend: deploy: placement: x-aws-role: \"arn:aws:iam::XXXXXXXXXXXX:role/foo\" x-aws-region: \"eu-west-3\" x-azure-region: \"france-central\"","title":"Example 1"},{"location":"docker/#example-2_3","text":"x-env: &env environment: - CONFIG_KEY - EXAMPLE_KEY services: first: <<: *env image: my-image:latest second: <<: *env image: another-image:latest In this example, the environment variables do not belong to either of the services. They\u2019ve been lifted out completely into the x-env extension field. This defines a new node which contains the environment field. The &env YAML anchor is used so both services can reference the extension field\u2019s value as *env .","title":"Example 2"},{"location":"docker/#example-3_3","text":"x-function: &function labels: function: \"true\" depends_on: - gateway networks: - functions deploy: placement: constraints: - 'node.platform.os == linux' services: ## Node.js gives OS info about the node (Host) nodeinfo: <<: *function image: functions/nodeinfo:latest environment: no_proxy: \"gateway\" https_proxy: $https_proxy ## Uses `cat` to echo back response, fastest function to execute. echoit: <<: *function image: functions/alpine:health environment: fprocess: \"cat\" no_proxy: \"gateway\" https_proxy: $https_proxy The nodeinfo and echoit services both include the x-function extension via the &function anchor, then set their specific image and environment.","title":"Example 3"},{"location":"docker/#example-4_1","text":"Using YAML merge it is also possible to use multiple extensions and share and override additional attributes for specific needs: x-environment: &default-environment FOO: BAR ZOT: QUIX x-keys: &keys KEY: VALUE services: frontend: image: example/webapp environment: << : [*default-environment, *keys] YET_ANOTHER: VARIABLE Note YAML merge only applies to mappings, and can't be used with sequences. In the example above, the environment variables are declared using the FOO: BAR mapping syntax, while the sequence syntax - FOO=BAR is only valid when no fragments are involved.","title":"Example 4"},{"location":"docker/#informative-historical-notes","text":"This section is informative. At the time of writing, the following prefixes are known to exist: Prefix Vendor/Organization docker Docker kubernetes Kubernetes","title":"Informative Historical Notes"},{"location":"docker/#specifying-byte-values","text":"Values express a byte value as a string in {amount}{byte unit} format: The supported units are b (bytes), k or kb (kilo bytes), m or mb (mega bytes) and g or gb (giga bytes). 2b 1024kb 2048k 300m 1gb","title":"Specifying byte values"},{"location":"docker/#specifying-durations","text":"Values express a duration as a string in the form of {value}{unit} . The supported units are us (microseconds), ms (milliseconds), s (seconds), m (minutes) and h (hours). Values can combine multiple values without separator. 10ms 40s 1m30s 1h5m30s20ms","title":"Specifying durations"},{"location":"docker/#interpolation","text":"Values in a Compose file can be set by variables and interpolated at runtime. Compose files use a Bash-like syntax ${VARIABLE} . Both $VARIABLE and ${VARIABLE} syntax is supported. Default values can be defined inline using typical shell syntax: ${VARIABLE:-default} evaluates to default if VARIABLE is unset or empty in the environment. ${VARIABLE-default} evaluates to default only if VARIABLE is unset in the environment. Similarly, the following syntax allows you to specify mandatory variables: ${VARIABLE:?err} exits with an error message containing err if VARIABLE is unset or empty in the environment. ${VARIABLE?err} exits with an error message containing err only if VARIABLE is unset in the environment. Interpolation can also be nested: ${VARIABLE:-${FOO}} ${VARIABLE?$FOO} ${VARIABLE:-${FOO:-default}} Other extended shell-style features, such as ${VARIABLE/foo/bar} , are not supported by Compose. You can use a $$ (double-dollar sign) when your configuration needs a literal dollar sign. This also prevents Compose from interpolating a value, so a $$ allows you to refer to environment variables that you don't want processed by Compose. web: build: . command: \"$$VAR_NOT_INTERPOLATED_BY_COMPOSE\" If Compose can't resolve a substituted variable and no default value is defined, it displays a warning and substitutes the variable with an empty string. As any values in a Compose file can be interpolated with variable substitution, including compact string notation for complex elements, interpolation is applied before a merge on a per-file basis. Interpolation applies only to YAML values, not to keys. For the few places where keys are actually arbitrary user-defined strings, such as labels or environment , an alternate equal sign syntax must be used for interpolation to apply. For example: services: foo: labels: \"$VAR_NOT_INTERPOLATED_BY_COMPOSE\": \"BAR\" services: foo: labels: - \"$VAR_INTERPOLATED_BY_COMPOSE=BAR\"","title":"Interpolation"},{"location":"docker/#merge-and-override","text":"Compose lets you define a Compose application model through multiple Compose files . When doing so, Compose follows the rules declared in this section to merge Compose files.","title":"Merge and override"},{"location":"docker/#mapping","text":"A YAML mapping gets merged by adding missing entries and merging the conflicting ones. Merging the following example YAML trees: services: foo: key1: value1 key2: value2 services: foo: key2: VALUE key3: value3 Results in a Compose application model equivalent to the YAML tree: services: foo: key1: value1 key2: VALUE key3: value3","title":"Mapping"},{"location":"docker/#sequence","text":"A YAML sequence is merged by appending values from the overriding Compose file to the previous one. Merging the following example YAML trees: services: foo: DNS: - 1.1.1.1 services: foo: DNS: - 8.8.8.8 Results in a Compose application model equivalent to the YAML tree: services: foo: DNS: - 1.1.1.1 - 8.8.8.8","title":"Sequence"},{"location":"docker/#exceptions","text":"","title":"Exceptions"},{"location":"docker/#shell-commands","text":"When merging Compose files that use the services attributes command , entrypoint and healthcheck: test , the value is overridden by the latest Compose file, and not appended. Merging the following example YAML trees: services: foo: command: [\"echo\", \"foo\"] services: foo: command: [\"echo\", \"bar\"] Results in a Compose application model equivalent to the YAML tree: services: foo: command: [\"echo\", \"bar\"]","title":"Shell commands"},{"location":"docker/#unique-resources","text":"Applies to the ports , volumes , secrets and configs services attributes. While these types are modeled in a Compose file as a sequence, they have special uniqueness requirements: Attribute Unique key volumes target secrets source configs source ports {ip, target, published, protocol} When merging Compose files, Compose appends new entries that do not violate a uniqueness constraint and merge entries that share a unique key. Merging the following example YAML trees: services: foo: volumes: - foo:/work services: foo: volumes: - bar:/work Results in a Compose application model equivalent to the YAML tree: services: foo: volumes: - bar:/work","title":"Unique resources"},{"location":"docker/#reset-value","text":"In addition to the previously described mechanism, an override Compose file can also be used to remove elements from your application model. For this purpose, the custom YAML tag !reset can be set to override a value set by the overriden Compose file. A valid value for attribute must be provided, but will be ignored and target attribute will be set with type's default value or null . For readability, it is recommended to explicitly set the attribute value to the null ( null ) or empty array [] (with !reset null or !reset [] ) so that it is clear that resulting attribute will be cleared. A base compose.yaml file: services: app: image: myapp ports: - \"8080:80\" environment: FOO: BAR And an overide.compose.yaml file: services: app: image: myapp ports: !reset [] environment: FOO: !reset null Results in: services: app: image: myapp","title":"Reset value"},{"location":"docker/#replace-value","text":"While !reset can be used to remove a declaration from a Compose file using an override file, !override allows you to fully replace an attribute, bypassing the standard merge rules. A typical example is to fully replace a resource definition, to rely on a distinct model but using the same name. A base compose.yaml file: services: app: image: myapp ports: - \"8080:80\" To remove the original port, but expose a new one, the following override file is used: services: app: ports: !override - \"8443:443\" This results in: services: app: image: myapp ports: - \"8443:443\" If !override had not been used, both 8080:80 and 8443:443 would be exposed as per the merging rules outlined above .","title":"Replace value"},{"location":"docker/#include","text":"A Compose application can declare dependency on another Compose application. This is useful if: - You want to reuse other Compose files. - You need to factor out parts of your application model into separate Compose files so they can be managed separately or shared with others. - Teams need to keep a Compose file reasonably complicated for the limited amount of resources it has to declare for it's own sub-domain, within a larger deployment. The include top-level section is used to define the dependency on another Compose application, or sub-domain. Each path listed in the include section is loaded as an individual Compose application model, with it's own project directory, in order to resolve relative paths. Once the included Compose application is loaded, all resources definitions are copied into the current Compose application model. Compose displays a warning if resource names conflict and doesn't try to merge them. To enforce this, include is evaluated after the Compose file(s) selected to define the Compose application model have been parsed and merged, so that conflicts between Compose files are detected. include applies recursively so an included Compose file which declares its own include section, triggers those other files to be included as well. Any volumes, networks, or other resources pulled in from the included Compose file can be used by the current Compose application for cross-service references. For example: include: - my-compose-include.yaml #with serviceB declared services: serviceA: build: . depends_on: - serviceB #use serviceB directly as if it was declared in this Compose file Compose also supports the use of interpolated variables with include . It's recommended that you specify mandatory variables . For example: include: -${INCLUDE_PATH:?FOO}/compose.yaml","title":"Include"},{"location":"docker/#short-syntax_6","text":"The short syntax only defines paths to other Compose files. The file is loaded with the parent folder as the project directory, and an optional .env file that is loaded to define any variables' default values by interpolation. The local project's environment can override those values. include: - ../commons/compose.yaml - ../another_domain/compose.yaml services: webapp: depends_on: - included-service ## defined by another_domain In the above example, both ../commons/compose.yaml and ../another_domain/compose.yaml are loaded as individual Compose projects. Relative paths in Compose files being referred by include are resolved relative to their own Compose file path, not based on the local project's directory. Variables are interpolated using values set in the optional .env file in same folder, and is overridden by the local project's environment.","title":"Short syntax"},{"location":"docker/#long-syntax_6","text":"The long syntax offers more control over the sub-project parsing: include: - path: ../commons/compose.yaml project_directory: .. env_file: ../another/.env","title":"Long syntax"},{"location":"docker/#path","text":"path is required and defines the location of the Compose file(s) to be parsed and included into the local Compose model. path can be set either to a string when a single Compose file is involved, or to a list of strings when multiple Compose files need to be merged together to define the Compose model to be included in the local application. include: - path: - ../commons/compose.yaml - ./commons-override.yaml","title":"path"},{"location":"docker/#project_directory","text":"project_directory defines a base path to resolve relative paths set in the Compose file. It defaults to the directory of the included Compose file.","title":"project_directory"},{"location":"docker/#env_file_1","text":"env_file defines an environment file(s) to use to define default values when interpolating variables in the Compose file being parsed. It defaults to .env file in the project_directory for the Compose file being parsed. env_file can be set either to a string or a list of strings when multiple environment files need to be merged to define a project environment. include: - path: ../another/compose.yaml env_file: - ../another/.env - ../another/dev.env The local project's environment has precedence over the values set by the Compose file, so that the local project can override values for customization.","title":"env_file"},{"location":"docker/#profiles_1","text":"With profiles you can define a set of active profiles so your Compose application model is adjusted for various usages and environments. The exact mechanism is implementation specific and may include command line flags, environment variables, etc. The services top-level element supports a profiles attribute to define a list of named profiles. Services without a profiles attribute are always enabled. A service is ignored by Compose when none of the listed profiles match the active ones, unless the service is explicitly targeted by a command. In that case its profile is added to the set of active profiles. Note All other top-level elements are not affected by profiles and are always active. References to other services (by links , extends or shared resource syntax service:xxx ) do not automatically enable a component that would otherwise have been ignored by active profiles. Instead Compose returns an error.","title":"Profiles"},{"location":"docker/#illustrative-example_1","text":"services: foo: image: foo bar: image: bar profiles: - test baz: image: baz depends_on: - bar profiles: - test zot: image: zot depends_on: - bar profiles: - debug In the above example: If the Compose application model is parsed with no profile enabled, it only contains the foo service. If the profile test is enabled, the model contains the services bar and baz , and service foo , which is always enabled. If the profile debug is enabled, the model contains both foo and zot services, but not bar and baz , and as such the model is invalid regarding the depends_on constraint of zot . If the profiles debug and test are enabled, the model contains all services; foo , bar , baz and zot . If Compose is executed with bar as the explicit service to run, bar and the test profile are active even if test profile is not enabled. If Compose is executed with baz as the explicit service to run, the service baz and the profile test are active and bar is pulled in by the depends_on constraint. If Compose is executed with zot as the explicit service to run, again the model is invalid regarding the depends_on constraint of zot , since zot and bar have no common profiles listed. If Compose is executed with zot as the explicit service to run and profile test is enabled, profile debug is automatically enabled and service bar is pulled in as a dependency starting both services zot and bar . See how you can use profiles in Docker Compose .","title":"Illustrative example"},{"location":"embedding/","text":"Embedding Embedding is a way to represent categorical variables in a way that can be used by machine learning algorithms. It is a way to represent a categorical variable as a continuous vector of numbers. This is done by creating a matrix of weights that is learned during the training process. The matrix is initialized randomly and updated during training. The embedding matrix is a dense representation of the categorical variable. Leaderboard https://huggingface.co/spaces/mteb/leaderboard","title":"embedding"},{"location":"embedding/#embedding","text":"Embedding is a way to represent categorical variables in a way that can be used by machine learning algorithms. It is a way to represent a categorical variable as a continuous vector of numbers. This is done by creating a matrix of weights that is learned during the training process. The matrix is initialized randomly and updated during training. The embedding matrix is a dense representation of the categorical variable.","title":"Embedding"},{"location":"embedding/#leaderboard","text":"https://huggingface.co/spaces/mteb/leaderboard","title":"Leaderboard"},{"location":"framework/","text":"Framework Pytorch vs Tensorflow The image range is different for each framework. In PyTorch, the image range is 0-1 while TensorFlow uses a range from 0 to 255. To use TensorFlow, we have to adapt the image range. To TF def dataset_to_tf( dataset, cols_to_retain, collate_fn, collate_fn_args, columns_to_np_types, output_signature, shuffle, batch_size, drop_remainder, ): \"\"\"Create a tf.data.Dataset from the underlying Dataset. This is a single-process method - the multiprocess equivalent is multiprocess_dataset_to_tf. Args: dataset (`Dataset`): Dataset to wrap with tf.data.Dataset. cols_to_retain (`List[str]`): Dataset column(s) to load in the tf.data.Dataset. It is acceptable to include column names that are created by the `collate_fn` and that do not exist in the original dataset. collate_fn(`Callable`): A function or callable object (such as a `DataCollator`) that will collate lists of samples into a batch. collate_fn_args (`Dict`): A `dict` of keyword arguments to be passed to the `collate_fn`. Can be empty. columns_to_np_types (`Dict[str, np.dtype]`): A `dict` mapping column names to numpy dtypes. output_signature (`Dict[str, tf.TensorSpec]`): A `dict` mapping column names to `tf.TensorSpec` objects. shuffle(`bool`): Shuffle the dataset order when loading. Recommended True for training, False for validation/evaluation. batch_size (`int`): Size of batches to load from the dataset. drop_remainder(`bool`, default `None`): Drop the last incomplete batch when loading. If not provided, defaults to the same setting as shuffle. Returns: `tf.data.Dataset` \"\"\" if config.TF_AVAILABLE: import tensorflow as tf else: raise ImportError(\"Called a Tensorflow-specific function but Tensorflow is not installed.\") getter_fn = partial( np_get_batch, dataset=dataset, cols_to_retain=cols_to_retain, collate_fn=collate_fn, collate_fn_args=collate_fn_args, columns_to_np_types=columns_to_np_types, return_dict=False, # TF expects numpy_function to return a list and will not accept a dict ) @tf.function(input_signature=[tf.TensorSpec(None, tf.int64)]) def fetch_function(indices): output = tf.numpy_function( getter_fn, inp=[indices], # This works because dictionaries always output in the same order Tout=[tf.dtypes.as_dtype(dtype) for dtype in columns_to_np_types.values()], ) return {key: output[i] for i, key in enumerate(columns_to_np_types.keys())} tf_dataset = tf.data.Dataset.from_tensor_slices(np.arange(len(dataset), dtype=np.int64)) if shuffle: tf_dataset = tf_dataset.shuffle(len(dataset)) tf_dataset = tf_dataset.batch(batch_size, drop_remainder=drop_remainder).map(fetch_function) def ensure_shapes(input_dict): return {key: tf.ensure_shape(val, output_signature[key].shape) for key, val in input_dict.items()} return tf_dataset.map(ensure_shapes) pytorch dataset to tf dataset import tensorflow as tf import torch # Assume that we have a PyTorch Dataset object called 'dataset' def pytorch_dataset_to_tensorflow_dataset(dataset): def generator(): for data in dataset: # Convert data from PyTorch tensors to TensorFlow tensors data = [tf.convert_to_tensor(x) for x in data] yield data # Create a TensorFlow Dataset from the generator dataset = tf.data.Dataset.from_generator(generator, output_types=data[0].dtype, output_shapes=data[0].shape) return dataset # Create a TensorFlow Dataset from the PyTorch Dataset dataset = pytorch_dataset_to_tensorflow_dataset(dataset) # Create a TensorFlow DataLoader from the TensorFlow Dataset dataloader = tf.data.DataLoader(dataset, batch_size=32, num_parallel_calls=tf.data.AUTOTUNE) image = tf.io.read_file(filename=filepath) image = tf.image.decode_jpeg(image, channels=3) #or decode_png The opposite of unsqueeze and squeeze is expand_dims : img = tf.expand_dims(img,axis=0) yield the desired/necessary transformations. As for the photos, I am quite sure that you missed a /255.0 in case of PyTorch or added a 255.0 division in case of TensorFlow. In fact, when digging deep into the Keras backend, you can see that when you call your preprocessing function, it will call this function here: def _preprocess_numpy_input(x, data_format, mode): \"\"\"Preprocesses a Numpy array encoding a batch of images. Arguments: x: Input array, 3D or 4D. data_format: Data format of the image array. mode: One of \"caffe\", \"tf\" or \"torch\". - caffe: will convert the images from RGB to BGR, then will zero-center each color channel with respect to the ImageNet dataset, without scaling. - tf: will scale pixels between -1 and 1, sample-wise. - torch: will scale pixels between 0 and 1 and then will normalize each channel with respect to the ImageNet dataset. Returns: Preprocessed Numpy array. \"\"\" if not issubclass(x.dtype.type, np.floating): x = x.astype(backend.floatx(), copy=False) if mode == 'tf': x /= 127.5 x -= 1. return x elif mode == 'torch': x /= 255. mean = [0.485, 0.456, 0.406] std = [0.229, 0.224, 0.225] else: if data_format == 'channels_first': # 'RGB'->'BGR' if x.ndim == 3: x = x[::-1, ...] else: x = x[:, ::-1, ...] else: # 'RGB'->'BGR' x = x[..., ::-1] mean = [103.939, 116.779, 123.68] std = None # Zero-center by mean pixel if data_format == 'channels_first': if x.ndim == 3: x[0, :, :] -= mean[0] x[1, :, :] -= mean[1] x[2, :, :] -= mean[2] if std is not None: x[0, :, :] /= std[0] x[1, :, :] /= std[1] x[2, :, :] /= std[2] else: x[:, 0, :, :] -= mean[0] x[:, 1, :, :] -= mean[1] x[:, 2, :, :] -= mean[2] if std is not None: x[:, 0, :, :] /= std[0] x[:, 1, :, :] /= std[1] x[:, 2, :, :] /= std[2] else: x[..., 0] -= mean[0] x[..., 1] -= mean[1] x[..., 2] -= mean[2] if std is not None: x[..., 0] /= std[0] x[..., 1] /= std[1] x[..., 2] /= std[2] return x mean and std mean = 0.0 std = 0.0 for images, _ in dl: batch_samples = images.size(0) # batch size (the last batch can have smaller size!) images = images.view(batch_samples, images.size(1), -1) mean += images.mean(2).sum(0) std += images.std(2).sum(0) mean /= len(dl.dataset) std /= len(dl.dataset) DataGenerator(keras.utils.Sequence): import numpy as np import keras class DataGenerator(keras.utils.Sequence): 'Generates data for Keras' def __init__(self, list_IDs, labels, batch_size=32, dim=(32,32,32), n_channels=1, n_classes=10, shuffle=True): 'Initialization' self.dim = dim self.batch_size = batch_size self.labels = labels self.list_IDs = list_IDs self.n_channels = n_channels self.n_classes = n_classes self.shuffle = shuffle self.on_epoch_end() def __len__(self): 'Denotes the number of batches per epoch' return int(np.floor(len(self.list_IDs) / self.batch_size)) def __getitem__(self, index): 'Generate one batch of data' # Generate indexes of the batch indexes = self.indexes[index*self.batch_size:(index+1)*self.batch_size] # Find list of IDs list_IDs_temp = [self.list_IDs[k] for k in indexes] # Generate data X, y = self.__data_generation(list_IDs_temp) return X, y def on_epoch_end(self): 'Updates indexes after each epoch' self.indexes = np.arange(len(self.list_IDs)) if self.shuffle == True: np.random.shuffle(self.indexes) def __data_generation(self, list_IDs_temp): 'Generates data containing batch_size samples' # X : (n_samples, *dim, n_channels) # Initialization X = np.empty((self.batch_size, *self.dim, self.n_channels)) y = np.empty((self.batch_size), dtype=int) # Generate data for i, ID in enumerate(list_IDs_temp): # Store sample X[i,] = np.load('data/' + ID + '.npy') # Store class y[i] = self.labels[ID] return X, keras.utils.to_categorical(y, num_classes=self.n_classes) class ImageDataGenerator(tf.keras.preprocessing.image.ImageDataGenerator): def __init__(self): super().__init__( rescale=1.0 / 255.0, rotation_range=10, width_shift_range=0.2, height_shift_range=0.2, zoom_range=[0.95, 1.05], shear_range=0.1, fill_mode=\"wrap\", horizontal_flip=True, vertical_flip=True, ) class Generator(object): def __init__(self, batch_size, name_x, name_y): data_f = None # h5py.File(open_directory, \"r\") self.x = data_f[name_x] self.y = data_f[name_y] if len(self.x.shape) == 4: self.shape_x = (None, self.x.shape[1], self.x.shape[2], self.x.shape[3]) if len(self.x.shape) == 3: self.shape_x = (None, self.x.shape[1], self.x.shape[2]) if len(self.y.shape) == 4: self.shape_y = (None, self.y.shape[1], self.y.shape[2], self.y.shape[3]) if len(self.y.shape) == 3: self.shape_y = (None, self.y.shape[1], self.y.shape[2]) self.num_samples = self.x.shape[0] self.batch_size = batch_size self.epoch_size = self.num_samples // self.batch_size + 1 * ( self.num_samples % self.batch_size != 0 ) self.pointer = 0 self.sample_nums = np.arange(0, self.num_samples) np.random.shuffle(self.sample_nums) def data_generator(self): for batch_num in range(self.epoch_size): x = [] y = [] for elem_num in range(self.batch_size): sample_num = self.sample_nums[self.pointer] x += [self.x[sample_num]] y += [self.y[sample_num]] self.pointer += 1 if self.pointer == self.num_samples: self.pointer = 0 np.random.shuffle(self.sample_nums) break x = np.array(x, dtype=np.float32) y = np.array(y, dtype=np.float32) yield x, y def get_dataset(self): dataset = tf.data.Dataset.from_generator( self.data_generator, output_signature=( tf.TensorSpec(shape=(), dtype=tf.int32), tf.RaggedTensorSpec(shape=(2, None), dtype=tf.int32), ), ) dataset = dataset.prefetch(1) return dataset def _load_image(self, image_path): image = cv2.imread(image_path) # BGR # image = cv2.cvtColor(image, cv2.COLOR_BGR2RGB) # image = tf.io.read_file(image_path) # image = tf.io.decode_image( # image, # channels=self.num_channels, # dtype=tf.dtypes.uint8, # expand_animations=False, # ) # image = tf.image.resize( # image, # self.dim, # method=tf.image.ResizeMethod.BILINEAR, # preserve_aspect_ratio=True, # antialias=False, # name=None, # ) # if not issubclass(image.dtype.type, np.floating): image = image.astype(np.float32) # image = image.astype(tf.keras.backend.floatx(), copy=False) image = self.apply_image_transforms(image) # 'RGB'->'BGR' # image = image[..., ::-1] # image = tf.image.convert_image_dtype(image, dtype=tf.uint8, saturate=False) # image = tf.cast(image, tf.float32) # / 127.5 # image -= 1.0 mean = [103.939, 116.779, 123.68] # mean_tensor = tf.keras.backend.constant(-np.array(mean)) # if tf.keras.backend.dtype(image) != tf.keras.backend.dtype(mean_tensor): # image = tf.keras.backend.bias_add( # image, # tf.keras.backend.cast(mean_tensor, tf.keras.backend.dtype(image)), # data_format=\"channels_last\", # ) # else: # image = tf.keras.backend.bias_add(image, mean_tensor, \"channels_last\") # image[0, :, :] -= mean[0] # image[1, :, :] -= mean[1] # image[2, :, :] -= mean[2] image[..., 0] -= mean[0] image[..., 1] -= mean[1] image[..., 2] -= mean[2] # image = tf.keras.applications.vgg16.preprocess_input(image) \"\"\" Preprocessed numpy.array or a tf.Tensor with type float32. The images are converted from RGB to BGR, then each color channel is zero-centered with respect to the ImageNet dataset, without scaling. \"\"\" return image Unfreeze specific layers Here is one way to unfreeze specific layers. We pick the same model and some layers (e.g. block14_sepconv2 ). The purpose is to unfreeze these layers and make the rest of the layers freeze . from tensorflow import keras base_model = keras.applications.Xception( weights='imagenet', input_shape=(150,150,3), include_top=False ) # free all layer except the desired layers # which is in [ ... ] for layer in base_model.layers: if layer.name not in ['block14_sepconv2', 'block13_sepconv1']: layer.trainable = False if layer.trainable: print(layer.name) block14_sepconv2 block13_sepconv1 Compute the trainable and non-trainable variables. import tensorflow.keras.backend as K import numpy as np trainable_count = np.sum([K.count_params(w) \\ for w in base_model.trainable_weights]) non_trainable_count = np.sum([K.count_params(w) \\ for w in base_model.non_trainable_weights]) print('Total params: {:,}'.format(trainable_count + non_trainable_count)) print('Trainable params: {:,}'.format(trainable_count)) print('Non-trainable params: {:,}'.format(non_trainable_count)) Total params: 20,861,480 Trainable params: 3,696,088 Non-trainable params: 17,165,392 tensorflow-macos Releases tensorflow-macos tensorflow-metal macOS version Features v2.5 0.1.2 12.0+ Pluggable device v2.6 0.2.0 12.0+ Variable sequences for RNN layers v2.7 0.3.0 12.0+ Custom op support v2.8 0.4.0 12.0+ RNN performance improvements v2.9 0.5.0 12.1+ Distributed training","title":"Framework"},{"location":"framework/#framework","text":"","title":"Framework"},{"location":"framework/#pytorch-vs-tensorflow","text":"The image range is different for each framework. In PyTorch, the image range is 0-1 while TensorFlow uses a range from 0 to 255. To use TensorFlow, we have to adapt the image range.","title":"Pytorch vs Tensorflow"},{"location":"framework/#to-tf","text":"def dataset_to_tf( dataset, cols_to_retain, collate_fn, collate_fn_args, columns_to_np_types, output_signature, shuffle, batch_size, drop_remainder, ): \"\"\"Create a tf.data.Dataset from the underlying Dataset. This is a single-process method - the multiprocess equivalent is multiprocess_dataset_to_tf. Args: dataset (`Dataset`): Dataset to wrap with tf.data.Dataset. cols_to_retain (`List[str]`): Dataset column(s) to load in the tf.data.Dataset. It is acceptable to include column names that are created by the `collate_fn` and that do not exist in the original dataset. collate_fn(`Callable`): A function or callable object (such as a `DataCollator`) that will collate lists of samples into a batch. collate_fn_args (`Dict`): A `dict` of keyword arguments to be passed to the `collate_fn`. Can be empty. columns_to_np_types (`Dict[str, np.dtype]`): A `dict` mapping column names to numpy dtypes. output_signature (`Dict[str, tf.TensorSpec]`): A `dict` mapping column names to `tf.TensorSpec` objects. shuffle(`bool`): Shuffle the dataset order when loading. Recommended True for training, False for validation/evaluation. batch_size (`int`): Size of batches to load from the dataset. drop_remainder(`bool`, default `None`): Drop the last incomplete batch when loading. If not provided, defaults to the same setting as shuffle. Returns: `tf.data.Dataset` \"\"\" if config.TF_AVAILABLE: import tensorflow as tf else: raise ImportError(\"Called a Tensorflow-specific function but Tensorflow is not installed.\") getter_fn = partial( np_get_batch, dataset=dataset, cols_to_retain=cols_to_retain, collate_fn=collate_fn, collate_fn_args=collate_fn_args, columns_to_np_types=columns_to_np_types, return_dict=False, # TF expects numpy_function to return a list and will not accept a dict ) @tf.function(input_signature=[tf.TensorSpec(None, tf.int64)]) def fetch_function(indices): output = tf.numpy_function( getter_fn, inp=[indices], # This works because dictionaries always output in the same order Tout=[tf.dtypes.as_dtype(dtype) for dtype in columns_to_np_types.values()], ) return {key: output[i] for i, key in enumerate(columns_to_np_types.keys())} tf_dataset = tf.data.Dataset.from_tensor_slices(np.arange(len(dataset), dtype=np.int64)) if shuffle: tf_dataset = tf_dataset.shuffle(len(dataset)) tf_dataset = tf_dataset.batch(batch_size, drop_remainder=drop_remainder).map(fetch_function) def ensure_shapes(input_dict): return {key: tf.ensure_shape(val, output_signature[key].shape) for key, val in input_dict.items()} return tf_dataset.map(ensure_shapes)","title":"To TF"},{"location":"framework/#pytorch-dataset-to-tf-dataset","text":"import tensorflow as tf import torch # Assume that we have a PyTorch Dataset object called 'dataset' def pytorch_dataset_to_tensorflow_dataset(dataset): def generator(): for data in dataset: # Convert data from PyTorch tensors to TensorFlow tensors data = [tf.convert_to_tensor(x) for x in data] yield data # Create a TensorFlow Dataset from the generator dataset = tf.data.Dataset.from_generator(generator, output_types=data[0].dtype, output_shapes=data[0].shape) return dataset # Create a TensorFlow Dataset from the PyTorch Dataset dataset = pytorch_dataset_to_tensorflow_dataset(dataset) # Create a TensorFlow DataLoader from the TensorFlow Dataset dataloader = tf.data.DataLoader(dataset, batch_size=32, num_parallel_calls=tf.data.AUTOTUNE) image = tf.io.read_file(filename=filepath) image = tf.image.decode_jpeg(image, channels=3) #or decode_png The opposite of unsqueeze and squeeze is expand_dims : img = tf.expand_dims(img,axis=0) yield the desired/necessary transformations. As for the photos, I am quite sure that you missed a /255.0 in case of PyTorch or added a 255.0 division in case of TensorFlow. In fact, when digging deep into the Keras backend, you can see that when you call your preprocessing function, it will call this function here: def _preprocess_numpy_input(x, data_format, mode): \"\"\"Preprocesses a Numpy array encoding a batch of images. Arguments: x: Input array, 3D or 4D. data_format: Data format of the image array. mode: One of \"caffe\", \"tf\" or \"torch\". - caffe: will convert the images from RGB to BGR, then will zero-center each color channel with respect to the ImageNet dataset, without scaling. - tf: will scale pixels between -1 and 1, sample-wise. - torch: will scale pixels between 0 and 1 and then will normalize each channel with respect to the ImageNet dataset. Returns: Preprocessed Numpy array. \"\"\" if not issubclass(x.dtype.type, np.floating): x = x.astype(backend.floatx(), copy=False) if mode == 'tf': x /= 127.5 x -= 1. return x elif mode == 'torch': x /= 255. mean = [0.485, 0.456, 0.406] std = [0.229, 0.224, 0.225] else: if data_format == 'channels_first': # 'RGB'->'BGR' if x.ndim == 3: x = x[::-1, ...] else: x = x[:, ::-1, ...] else: # 'RGB'->'BGR' x = x[..., ::-1] mean = [103.939, 116.779, 123.68] std = None # Zero-center by mean pixel if data_format == 'channels_first': if x.ndim == 3: x[0, :, :] -= mean[0] x[1, :, :] -= mean[1] x[2, :, :] -= mean[2] if std is not None: x[0, :, :] /= std[0] x[1, :, :] /= std[1] x[2, :, :] /= std[2] else: x[:, 0, :, :] -= mean[0] x[:, 1, :, :] -= mean[1] x[:, 2, :, :] -= mean[2] if std is not None: x[:, 0, :, :] /= std[0] x[:, 1, :, :] /= std[1] x[:, 2, :, :] /= std[2] else: x[..., 0] -= mean[0] x[..., 1] -= mean[1] x[..., 2] -= mean[2] if std is not None: x[..., 0] /= std[0] x[..., 1] /= std[1] x[..., 2] /= std[2] return x","title":"pytorch dataset to tf dataset"},{"location":"framework/#mean-and-std","text":"mean = 0.0 std = 0.0 for images, _ in dl: batch_samples = images.size(0) # batch size (the last batch can have smaller size!) images = images.view(batch_samples, images.size(1), -1) mean += images.mean(2).sum(0) std += images.std(2).sum(0) mean /= len(dl.dataset) std /= len(dl.dataset)","title":"mean and std"},{"location":"framework/#datageneratorkerasutilssequence","text":"import numpy as np import keras class DataGenerator(keras.utils.Sequence): 'Generates data for Keras' def __init__(self, list_IDs, labels, batch_size=32, dim=(32,32,32), n_channels=1, n_classes=10, shuffle=True): 'Initialization' self.dim = dim self.batch_size = batch_size self.labels = labels self.list_IDs = list_IDs self.n_channels = n_channels self.n_classes = n_classes self.shuffle = shuffle self.on_epoch_end() def __len__(self): 'Denotes the number of batches per epoch' return int(np.floor(len(self.list_IDs) / self.batch_size)) def __getitem__(self, index): 'Generate one batch of data' # Generate indexes of the batch indexes = self.indexes[index*self.batch_size:(index+1)*self.batch_size] # Find list of IDs list_IDs_temp = [self.list_IDs[k] for k in indexes] # Generate data X, y = self.__data_generation(list_IDs_temp) return X, y def on_epoch_end(self): 'Updates indexes after each epoch' self.indexes = np.arange(len(self.list_IDs)) if self.shuffle == True: np.random.shuffle(self.indexes) def __data_generation(self, list_IDs_temp): 'Generates data containing batch_size samples' # X : (n_samples, *dim, n_channels) # Initialization X = np.empty((self.batch_size, *self.dim, self.n_channels)) y = np.empty((self.batch_size), dtype=int) # Generate data for i, ID in enumerate(list_IDs_temp): # Store sample X[i,] = np.load('data/' + ID + '.npy') # Store class y[i] = self.labels[ID] return X, keras.utils.to_categorical(y, num_classes=self.n_classes) class ImageDataGenerator(tf.keras.preprocessing.image.ImageDataGenerator): def __init__(self): super().__init__( rescale=1.0 / 255.0, rotation_range=10, width_shift_range=0.2, height_shift_range=0.2, zoom_range=[0.95, 1.05], shear_range=0.1, fill_mode=\"wrap\", horizontal_flip=True, vertical_flip=True, ) class Generator(object): def __init__(self, batch_size, name_x, name_y): data_f = None # h5py.File(open_directory, \"r\") self.x = data_f[name_x] self.y = data_f[name_y] if len(self.x.shape) == 4: self.shape_x = (None, self.x.shape[1], self.x.shape[2], self.x.shape[3]) if len(self.x.shape) == 3: self.shape_x = (None, self.x.shape[1], self.x.shape[2]) if len(self.y.shape) == 4: self.shape_y = (None, self.y.shape[1], self.y.shape[2], self.y.shape[3]) if len(self.y.shape) == 3: self.shape_y = (None, self.y.shape[1], self.y.shape[2]) self.num_samples = self.x.shape[0] self.batch_size = batch_size self.epoch_size = self.num_samples // self.batch_size + 1 * ( self.num_samples % self.batch_size != 0 ) self.pointer = 0 self.sample_nums = np.arange(0, self.num_samples) np.random.shuffle(self.sample_nums) def data_generator(self): for batch_num in range(self.epoch_size): x = [] y = [] for elem_num in range(self.batch_size): sample_num = self.sample_nums[self.pointer] x += [self.x[sample_num]] y += [self.y[sample_num]] self.pointer += 1 if self.pointer == self.num_samples: self.pointer = 0 np.random.shuffle(self.sample_nums) break x = np.array(x, dtype=np.float32) y = np.array(y, dtype=np.float32) yield x, y def get_dataset(self): dataset = tf.data.Dataset.from_generator( self.data_generator, output_signature=( tf.TensorSpec(shape=(), dtype=tf.int32), tf.RaggedTensorSpec(shape=(2, None), dtype=tf.int32), ), ) dataset = dataset.prefetch(1) return dataset def _load_image(self, image_path): image = cv2.imread(image_path) # BGR # image = cv2.cvtColor(image, cv2.COLOR_BGR2RGB) # image = tf.io.read_file(image_path) # image = tf.io.decode_image( # image, # channels=self.num_channels, # dtype=tf.dtypes.uint8, # expand_animations=False, # ) # image = tf.image.resize( # image, # self.dim, # method=tf.image.ResizeMethod.BILINEAR, # preserve_aspect_ratio=True, # antialias=False, # name=None, # ) # if not issubclass(image.dtype.type, np.floating): image = image.astype(np.float32) # image = image.astype(tf.keras.backend.floatx(), copy=False) image = self.apply_image_transforms(image) # 'RGB'->'BGR' # image = image[..., ::-1] # image = tf.image.convert_image_dtype(image, dtype=tf.uint8, saturate=False) # image = tf.cast(image, tf.float32) # / 127.5 # image -= 1.0 mean = [103.939, 116.779, 123.68] # mean_tensor = tf.keras.backend.constant(-np.array(mean)) # if tf.keras.backend.dtype(image) != tf.keras.backend.dtype(mean_tensor): # image = tf.keras.backend.bias_add( # image, # tf.keras.backend.cast(mean_tensor, tf.keras.backend.dtype(image)), # data_format=\"channels_last\", # ) # else: # image = tf.keras.backend.bias_add(image, mean_tensor, \"channels_last\") # image[0, :, :] -= mean[0] # image[1, :, :] -= mean[1] # image[2, :, :] -= mean[2] image[..., 0] -= mean[0] image[..., 1] -= mean[1] image[..., 2] -= mean[2] # image = tf.keras.applications.vgg16.preprocess_input(image) \"\"\" Preprocessed numpy.array or a tf.Tensor with type float32. The images are converted from RGB to BGR, then each color channel is zero-centered with respect to the ImageNet dataset, without scaling. \"\"\" return image","title":"DataGenerator(keras.utils.Sequence):"},{"location":"framework/#unfreeze-specific-layers","text":"Here is one way to unfreeze specific layers. We pick the same model and some layers (e.g. block14_sepconv2 ). The purpose is to unfreeze these layers and make the rest of the layers freeze . from tensorflow import keras base_model = keras.applications.Xception( weights='imagenet', input_shape=(150,150,3), include_top=False ) # free all layer except the desired layers # which is in [ ... ] for layer in base_model.layers: if layer.name not in ['block14_sepconv2', 'block13_sepconv1']: layer.trainable = False if layer.trainable: print(layer.name) block14_sepconv2 block13_sepconv1","title":"Unfreeze specific layers"},{"location":"framework/#compute-the-trainable-and-non-trainable-variables","text":"import tensorflow.keras.backend as K import numpy as np trainable_count = np.sum([K.count_params(w) \\ for w in base_model.trainable_weights]) non_trainable_count = np.sum([K.count_params(w) \\ for w in base_model.non_trainable_weights]) print('Total params: {:,}'.format(trainable_count + non_trainable_count)) print('Trainable params: {:,}'.format(trainable_count)) print('Non-trainable params: {:,}'.format(non_trainable_count)) Total params: 20,861,480 Trainable params: 3,696,088 Non-trainable params: 17,165,392","title":"Compute the trainable and non-trainable variables."},{"location":"framework/#tensorflow-macos-releases","text":"tensorflow-macos tensorflow-metal macOS version Features v2.5 0.1.2 12.0+ Pluggable device v2.6 0.2.0 12.0+ Variable sequences for RNN layers v2.7 0.3.0 12.0+ Custom op support v2.8 0.4.0 12.0+ RNN performance improvements v2.9 0.5.0 12.1+ Distributed training","title":"tensorflow-macos Releases"},{"location":"git/","text":"GIT README.md \ud83d\udc4b Hi, I\u2019m @furyhawk \ud83d\udc40 I\u2019m interested in AI \ud83c\udf31 I\u2019m currently learning AI \ud83d\udc9e\ufe0f I\u2019m looking to collaborate on AI \ud83d\udceb How to reach me ... https://github.com/furyhawk git config --global user.name \"furyhawk\" git config --global user.email furyx@hotmail.com git config pull.rebase true commit msg : \u2502 \u2502 \u2502 \u2514\u2500\u2af8 Summary in present tense. Not capitalized. No period at the end. \u2502 \u2514\u2500\u2af8 Commit Type: build|cicd|docs|feat|fix|node|refactor|test Fetch/pull all branches git branch -r | grep -v '\\->' | sed \"s,\\x1B\\[[0-9;]*[a-zA-Z],,g\" | while read remote; do git branch --track \"${remote#origin/}\" \"$remote\"; done git fetch --all git pull --all How can I enable github notifications? install and authenticate with the github cli: pacman -S github-cli gh auth login ubuntu: type -p curl >/dev/null || (sudo apt update && sudo apt install curl -y) curl -fsSL https://cli.github.com/packages/githubcli-archive-keyring.gpg | sudo dd of=/usr/share/keyrings/githubcli-archive-keyring.gpg \\ && sudo chmod go+r /usr/share/keyrings/githubcli-archive-keyring.gpg \\ && echo \"deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/githubcli-archive-keyring.gpg] https://cli.github.com/packages stable main\" | sudo tee /etc/apt/sources.list.d/github-cli.list > /dev/null \\ && sudo apt update \\ && sudo apt install gh -y reset to last working commit: git reset --hard git push --force working with submodules: git submodule update --init --recursive There are four steps involved when you delete a submodule. # 1. deinit the submodule git submodule deinit # 2. Remove the submodule directory from Git git rm # 3. Remove the submodule directory from .git/modules/ rm -rf .git/modules/ # 4. commit and push the changes # add submodule and define the master branch as the one you want to track git submodule add -b master [URL to Git repo] git submodule init","title":"git"},{"location":"git/#git","text":"","title":"GIT"},{"location":"git/#readmemd","text":"\ud83d\udc4b Hi, I\u2019m @furyhawk \ud83d\udc40 I\u2019m interested in AI \ud83c\udf31 I\u2019m currently learning AI \ud83d\udc9e\ufe0f I\u2019m looking to collaborate on AI \ud83d\udceb How to reach me ... https://github.com/furyhawk git config --global user.name \"furyhawk\" git config --global user.email furyx@hotmail.com git config pull.rebase true","title":"README.md"},{"location":"git/#commit-msg","text":": \u2502 \u2502 \u2502 \u2514\u2500\u2af8 Summary in present tense. Not capitalized. No period at the end. \u2502 \u2514\u2500\u2af8 Commit Type: build|cicd|docs|feat|fix|node|refactor|test","title":"commit msg"},{"location":"git/#fetchpull-all-branches","text":"git branch -r | grep -v '\\->' | sed \"s,\\x1B\\[[0-9;]*[a-zA-Z],,g\" | while read remote; do git branch --track \"${remote#origin/}\" \"$remote\"; done git fetch --all git pull --all","title":"Fetch/pull all branches"},{"location":"git/#how-can-i-enable-github-notifications","text":"install and authenticate with the github cli: pacman -S github-cli gh auth login ubuntu: type -p curl >/dev/null || (sudo apt update && sudo apt install curl -y) curl -fsSL https://cli.github.com/packages/githubcli-archive-keyring.gpg | sudo dd of=/usr/share/keyrings/githubcli-archive-keyring.gpg \\ && sudo chmod go+r /usr/share/keyrings/githubcli-archive-keyring.gpg \\ && echo \"deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/githubcli-archive-keyring.gpg] https://cli.github.com/packages stable main\" | sudo tee /etc/apt/sources.list.d/github-cli.list > /dev/null \\ && sudo apt update \\ && sudo apt install gh -y reset to last working commit: git reset --hard git push --force working with submodules: git submodule update --init --recursive There are four steps involved when you delete a submodule. # 1. deinit the submodule git submodule deinit # 2. Remove the submodule directory from Git git rm # 3. Remove the submodule directory from .git/modules/ rm -rf .git/modules/ # 4. commit and push the changes # add submodule and define the master branch as the one you want to track git submodule add -b master [URL to Git repo] git submodule init","title":"How can I enable github notifications?"},{"location":"gpt/","text":"gpt https://github.com/furyhawk/nanoGPT train python train.py --dataset=shakespeare --n_layer=4 --n_head=4 --n_embd=64 --device=mps --compile=False --eval_iters=1 --block_size=64 --batch_size=16 logs iter 432061: loss 3.3774, time 102.72ms number of parameters: 3.42M No meta.pkl found, assuming GPT-2 encodings... The state of heaven with him yet have set The enam received in his power, It will be such,morrow; if he be By common enemies, so may not do, you As if you mark the truth. FLORIZEL: Why is my part; Not here my life, we'll do't, where it is The lark that the Earl of Warwick is slain: On bold, my rest, my liege, in whose good time I'll drown him worthy me to my Earl towards, Is Come to draw to scARD a fearful day. 3 KING HENRY VI KING LEWIS XI: What! canst thou live in poor soul, To do him banish'd at the same our point And the freshinks I come, 'tis aught to have but I fear you joys: all that I cannot On thee would of my heart, my poor soul, To be so bold a knave in this time; And thou by, she's children's blood, only, it is To see my life. MARIANA: A thousand part, indeed, sir, a time As you will wonder not: come, my liege, I well; And he is too a gentleman. See that with the voice that at once, you must Flried the senators of the any it good to us being son: but, I cannot be done; but we can tell him; for him I'll draw your words together. What you will know how you? COMINIUS: They are, uncle. CORIOLANUS: Therefore, my gracious friend, I am the farther un ches: What, poor! I beseech you, let it be a cousin. VOLUMNIA: Tush, the gods! MENENIUS: I am too! Nay begin to bid my lady play well: I am no pity that willy heir is right; But, by his leave I have or why it is. CORIOLANUS: Not? let me see, you are strange? They have made no more than all. Hath my wife's lord, upon pain of death. Third Citizen: Why, I shall have had some heard of you hear. QUEEN MARGARET: And so we all, I thank thee: 'tis more; Thou art the letter, and thou hast not so: wouldst be done, hark you, I know, my soul here have t something So to say, but I pray, and no more. ISABELLA: O, if it were full of such deep Norfolk, love, I would have more than his gracious fault Than Angelo first to you at great comfort. ANGELO: Yea, and make you he crook 'pt TrleKE: I'll make. Provost: What, my general? DUKE VINCENTIO: 'Tis like of his most, And leave me to the will for my poor side. ANGELO: Hath the king'ster heaven with the majesty of tears, With a kind: I have, my lord, must die. MARCIUS: Tell me hate them not, honesty are there To entreat of my wife, and my son to my order, And set the mother all my father's consent To make the must fly: so go: From whom that I would not, I do not fear, As well I will have me, you are vile toward. QUEEN MARGARET: But, by your vouchius, I fear not what I am: For since we came, your father's, at the least? Second Servant: Tis more than this of you can but have done My love or mercy to the queen of heaven: But I have forgot it, more, goodenio. CAPULET: Ay, as the matter I told you speak, my lord. BALTHASAR: But to the case of the Tower. ROMEO: Never; live, my lord, my soul is as well: I hope, by this which is a villain, When that thy brothers Romeo is, one sun With thy mother's wife, that I may return To know me what a dream hath as well As she run thought to be your mother, I I may not need to my sweet, CAPULET: Ay, what art thou? ROMEO: A queen, nurse? JULIET: Good lords, If thou dost know ye but. JULIET: Say, that Richmond's soul, take me in all. ROMEO: O, Romeo, I will not weep a while. JULIET: For the defry of grief makes them fair: But since unsapblful villain's wit of heaven It was a friend for ever-day, When I was lawful oath and not so, I am born to do it would it might be, I am revenged on-hearted my leaves dead king's oath, For the wantless blood whose heads must enter: 'Tis the Lord Northumberland that King Edward was himself, The Duke of Warwick, and his place in France, And thou shalt come to my will-day shall not stay. HENRY BOLINGBROKE: Why, howof, I beseech you talk of, Here must I play my heart with a cruelay; That doth not fall dead, I know the matter. DUKE VINCENTIO: You had even like'd, fie me, 'tis, I am resolved: Nay, unadBe now, I have heard her speak more than you I speak at peace. ANGELO: Well, I do beseech you, fair sir. Your eyes will not come upon me: he is most man, it is not so: so most much little most wrong, and may go to that by: which you have therein your hate, we may see themselves. LADY CAPULET: We are in prison, one word; ask him; For some you win me, upon his best: You know no better for the mind, there's no matter stands on him. DUCHESS OF YORK: How wouldst thou? Messenger: My gracious lord, I like youThis. DUKE OF YORK: Is it not dead, cousin. DUCHESS OF YORK: Then 'tis shame forAnd all that thou wast? KING R DUKE VINCENTIO: Romeo. Provost: It do trust me. ANGELO: Our lord, I cannot speak the good word: Since I havewas not it, I do'll therefore know Will a divineade me speak. ISABELLA: O, let me hear what you did deliver, By 'twere pity to him. This friends, be his head it, with one that's no matter, look to me as this lure of the prince a curse; more stands on't; No, sir; not a very weak of mine, If not that. Now, andso death's like death! I see, and hear Montague, my Montague. Who knows thou nothing; be not gentle, I may not, yet they shall be so, then; And had he come'd withAnd the oracle: I'll have your heart: that I shall follow that send him; that, we will warrant you, andPEY? ELBOW: Nay, he shall get a man to me. ELBOW: Look, very well, he shallEL' the better royal, which I do do't But I shall find, a time will be from his spirit. I prithee, take one: give me not your comfortier: I will not be come to-morrow; let's entreaty? ROMEO: I beseech your grace, sir,-- BISHOP OF CARke, my dTwere 't. BUCKINGHAM: I will, my lord: nay, You will have you go with me to please again. BUCKINGHAM: You will not hear duty, but for I will, sir, You have not the lady will with thee love But what you may! You are sure of he is to be a doth valiant; I will wot the prince: and have first to-morrow then. ANGELO: Thy business is the more and that my brother's sister. ISABELLA: It cannot be so. DUKE VINCENTIO: I say you, well; for if his present is the WARWICK: And thou wert enam'd; I'll not be mine own? KING LEWIS XI: What man is none than that at his house, That bear their down heart to Edward's-t thou him? WARWICK: Bid me, noble lord, our friends are fled, And sit to be true; and with all our sonsiss, From forth the most defend in theile departedath Be not born to make a husband as if I send it to thought that I did weep, And quench blood or not well. KING RICHARD II: O I was Clarence! What'sFL words were it: That I, as we are, that they have no friend, Even in the blood of heaven of fight, Thou p country's blood and'd. Come, and go with thee go; And, as mine I remember as any man's son As merry as I told the people and am mighty As'd an hour begin and bears our hands, But shame no other Paris and her be drawn. ROMEO: Is my long come. carry, let'st thou father In this remembrance of love: thou eat not 'em most noble swear tongue for what thou mayst, not wert between. The Earl of heaven of Warwick's love, this man, That's hearts, the true king, myMON and Sir. And unlook'd in the princess slain, Is beg of hen, and did yield in their arms At no moreOR create give from their hands, For the dead George of Clarence to his king, Who spake me his enemies shall hold it. BUCKINGHAM: My lord, my gracious lord, You had a power of wisdom cam thou out of mine? KING RICHARD II: He hath enem this, people, but I'll win me. CATESBY: Then here, my lord. KING RICHARD III: Nor I, the crown that queen is slain, To take the devil of our other linean ground, Not his our kingly curse people's-Which they that Is put to have open honesty Either to come in their arms. QUEEN MARGARET: ANTIGONUS: Hear you not? First Citizen: Come, sir; go, be it must none but you: I am ta'en and leave of you; and, then, go with me. First Servingman: Why, we will, sir. Second Servingman: I would not, no more: the provost. Third Servingman: What's the matter? Third Servingman: What's the matter? Second Servingman: No, my master, I can tell; I know how it you, thereof he m it. Clown: He's here; we should be a witness to the purpose. Third Citizen: He hath done what you are done. Third Servingman: A Romeo, sir, for them he would have found baw Rome of the world. CORIOLANUS: But I love the child, Not that thou hast other of thy life. Third Servingman: If thou she were thrice a man, And had these griefing force royal royal queen; And I am son, and I love thee myself. KING RICHARD III: But I will be poor; which he is he made? BUCKINGHAM: My gracious lord, let's see your grace be company. KING RICHARD III: Why, what a fellow should be? BUCKINGHAM: My lord, 'tis a subject, proud Which else, which, had some him, it will with heaven Till wind manage our doth made over his land. Why, proud I have ta'en the service of the people, That would not have more better that, Which will be satisfied; and, how his ignorant shouldER' He seems not? and I have had rather thy, To do my country, to make the'd right, With all the envy of this loyal, Our holy and be England and Duke of York, Not in his second soul I and have been The which never bid us, and call King Richard's lord. HENRY BOLINGBROKE: Welcome, uncle; we shall not be unto him; But so, my good son, I I do further. KING RICHARD III: Why, then, I confess thee, lord, If thou hadst never yet thou yet hadst never, But yet thou wilt wert up in thy horse. CLIFFORD: Clarence and Gloucester, I will not bethink, But that's not yet did nothing: but the gods The times of revenge! RICHARD: Nay, bring me what ancient, and this hand.' KING HENRY VI: But, to thee speak commonth very day! CLIFFORD: That thou wert so disgraced me to me. KING HENRY VI: Woth he the oath that I should tread upon our way? CLIFFORD: What may your grace in poor Henry's life and him? WARWICK: No, like a bawd, you not so; and, and for the poor one did, he is even to give his l. WARWICK: Uncle, so: I do intend to him in all, I hope. KING HENRY VI: How far I mean? if this be so bold with! The which, his queen, the queen, his love; And so, gentle king, may they do you in at night And fly him in the slander of his king, To be revenged on him that, so should you have; For, by this way be you found you to, She's a woman. ANGELO: He will not see you, sir. ESCALUS: I am aTo your request in hand, you shall in arms You must give, and yet go by, a course it is thence. ESCALUS: LARTI not mock mine own good. ANGELO: Do you hear. The duke hath forgot you for what you are? ESCALUS: BUS: for whose offence? CORIOLANUS: What to you? CORIOLANUS: O, worthy madam, And have I too most, you shall not. AUFIDIUS: Worthy sir, farewell. VOL Had not himself, nor known she! Provost: An me, both that; he did, sir, to steal-on Her mother's Romeo! O old faithful friar! JULIET: Why do I more than that I should never speak of. ROMEO: Nay, rather give me leave me that I will; For I shall find some better happy days Than lay, a man. FRIAR LAURENCE: But slain, her hath the next day a little. ROMEO: Nay, good woman; my turn in thy sake art full nights here. Who is dead and young prince. Ah, how, dost thou find me to thy breast, Thine hast he, and more, a noble fawd. Sir, thou canst not speak, I wouldst thou wert For sleep the mark of thy deadly years? Ah, keep'st, asLord men, thou hast got By so thy voice: there's no power, it is none honest. MERCUTIO: Tybalt, Romeo, whom I have the best I think how I have done. ROMEO: Thy life art thou, that thou art too fair! BENVOLIO: Mehe, and go not in 't. MERCUTIO: Nay, I'll bring some noise for a for that. CLAUDIO: Your face? aarer is a traitor's head to pluck him to the king the house. But which you can do I have to the hand; for it will be, as I am and a block withFor a man that Claudio hath married, If be a uncle. DUKE VINCENTIO: He did know more. You are most little more, he hath done it well, our the city and do them to make me theD:' Aufidius come to me and brother; and she isome with those that hath been, 'twere pity us all. Then, soft, wot each that am aly; For now I did well know. My mother! LEONTES: What's well man? CAMILLO: KING RICHARD III: Be not so? HENRY BOLINGBROKE: So that of God he would have the king his! BUCKINGHAM: I'll make more than my gracious case: I will bear the deed. KING RICHARD III: tis you by king, I would am I bethink, To make an amissant in love That thou wouldst protest to keep post toOR'n; And thou he is not nowSo. LEONTES: Unman, it is; Nay, good my good lord, be satisfied still. LEONTES: A most business: Let me have been since I was to be mercy; Whiles the lark-f orth-she, He cannot, Marcius worthyKE: 'tis well stock no poison. The manner of the envy he spake to Marcius, That with the power is worth dead, the doth be brief, In worthy Romeo hastous tune up. KING RICHARD II: Whom was it so, my child is set on death. HENRY BOLINGBROKE: ClOR, my lord; With late that cons news, for, to that I would Trueass the great number pardon'd, from your head already're his mind i' the there, I will. DUKE VINCENTIO: You do but see your highness of your arms Would know the king's mouth with peace. MARIANA: I mean, a son, it is not a word through a tear for Claudio, and be banished; But when you bid this manner bless us, In whom we have had been wont to do, 'twas upon the business, you I shall call'd his friends to him, and then. DUKE OF Lord: He's tearsark; for long my gentleness' I let me hear; good cousin, adieu! Belike his hands I in his remedy; And I have died to face, my love to me, And I will make him leave of can say, I had rather keep their words with him to keep him Whither, to thy fortune but put off; And see that this terror may be so, So if that cannot: so that's the friend, That will be deservedly of your honour's life, Let thy rage here cut in his grave; One side do not. MENENIUS: Hath the people's great state by power, That they shall have set up a part; But, as minetis the nothing of a man, That is the deputy,tis well for fault, And breathed his honour with the fear of death, That all for that as you are, and we may live. BALTHASAR: A pretty fardman! RICHMOND: Bid him be here? and I'll make me pardon up. RIVERS: What, hast he not? BUCKINGHAM: My lord, my gracious lord. KING RICHARD III: As I remember; against thee, thou wert not so much done, That thou canst swear and thee, for thou hast no cause. RATCLIFF: I will not be not, my lord. KING RICHARD III: Ay, O my children, my eyes! BUCKINGHAM: No, mighty lord, I hope, shall understand it. KING RICHARD III: I know no? but himself he is done; The side must be seen and make me wrong. QUEEN ELIZABETH: That thou mean I sent to see thy life To have no more fit than it is in this world To look upon thy brother's make me hear, The Duke of Norfolk; if thou darest with Thy Lord Northumberland, rouse'd and thy crown, For thou shalt no life but by thy kingdom. KING EDWARD IV: But thou, in my turn, my grave jest thou, You, the newats, and have the ears, Where fruit men have done-bended, then my daughter Of happyness! O my woman! If thou didst, thou tis'st fair a piteous, which he was, And know not 't prove a thousand right while you That you are to have something: so, my kingdom, 'Tis just that I iter 506998: loss 3.5516, time 71.91ms iter 506999: loss 3.6803, time 72.24ms iter 507000: loss 3.3026, time 72.24ms iter 507001: loss 3.4185, time 71.62ms iter 507002: loss 3.6271, time 73.12ms iter 507003: loss 3.5836, time 71.73ms iter 507004: loss 3.1981, time 75.92ms iter 507005: loss 3.6141, time 71.98ms iter 507006: loss 3.2294, time 73.99ms number of parameters: 3.42M No meta.pkl found, assuming GPT-2 encodings... The state of heaven with him, and that his body is humray'd this. WARWICK: And I. DUCHESS OF YORK: My dear lord, DUCHESS OF YORK: HENRY BOLINGBROKE: As far go we as any we toOL? HORTENSIO: I think she is well: then will you have so: I would we were not been then in war's good. DUKE VINCENTIO: We shall not be still be gone and been to see. GREMIO: And your good lord, I would all and like you; and in your hope. ISABELLA: I can know the truth, I am your father's house. DUKE VINCENTIO: It is no gone, sir? LUCIO: I think I would, sir, I know say, I am a friendlike by him: it is the very much of your blood; if Would I be well show'd of you in the bosom And our new-a. Provost: MAMILLIUS: My lord, I'll warrant thee a holy heart: which's with mine, I am too good to him. LEONTES: I am none; Sir, for your son I have here in this business To the justice of your father's worth. PAULINA: It is my gr in that I were, To the most of your own: your therefore have Be consul: Come, come, go, go, be much here too much: I am no more. LEONTES: They are such a and the and yet. HERMIONE: The better, sir; I am an't that you have notity. LEONTES: LEONTES: Well, well, what! LARTIUS: I am so. PAULINA: I pray any I, sir? PAULINA: Alas, poor woman! What ha HENRYHAM: Too true Clarence was a holy bed. CAMILLO I shall have had a brother of your son, To yield your brother's brother. LordOUCESTER: CLARENCE: KING EDWARD IV: KING EDWARD IV: Why, then, think you what: go-morrow, son, That you must not do that I am just, For when you can, my lord, my brother, Henry With your brother, if he bring end to your last. And, high-ed, if you see her you do, He would not have been thus an whose speak With that most un Butice: I would, he were born, a name: The house of him I'll speak to see your wife's. DUKE VINCENTIO: 'Tis not her in, sir, his friends, which Of wrongstAName to the earth, with a little way. ISABELLA: I am so mither'd by the point of it That I have had. DUKE VINCENTIO: O, let me think; and let me see a word. Provost: I wouldISABELLA: Wilt thou not, then, not in a very kind for a very for me that I did use me from him to speak. And I beseech your majesty and your bosom And my great opinion, have a wife and father, Unto my brother father and my well. FRIAR LAURENCE: And there is my hNor talk'd be no sworn to death But that I should think 'twack the part of you In the duke's daughter. Nurse: A love a man that his blood is more a life Than this may standow make her false friends. Nay, more farewell; I not for't; Our unf shors'll ask you: come by the man-likeent of this lady is the least, But to the everatter of me dearest day, Or me but one earth, like a thing that Not so much sorrow's, as many more is true Than this in his own! as our bthe light As I do! What's in all my shame already may in openings! You, my sweet HASTINGS: Be early what would you say to our lord? YORK: It is not so. GLOUCESTER: It is. LADY ANNE: sweet my old grace, that thou follow'st! GLOUCESTER: And, for God's sake, till you are three years. LADY ANNE: What, prince? GLOUCESTER: No noble thing I should be a happy king? GLOUCESTER: Away with him! DUCHESS OF YORK: My lord, I must not need: Yere to my children's love's loss of thee, But thou, even to the king, thou thy doubt'st tongue. KING RICHARD III: What is't o'So? DUCHESS OF YORK: I, by my life, I have done, but so in once For in the virtue of our blood: without that kind Was never so far gGLOUCESTR well, And I will follow him. DUKE VINCENTIO: M King Richard, give me leave to my thou thou go Be at home of these good fairWARD. But of what, thou dost to do Edward! B vouch'd: My lords, more than you shall please our royal name. LORD ROMASET: So, to save the king; yet he's just, And come, to answer me in him my country, And you shall have you where we did say the prince? DUCHESS OF YORK: My Lord, may you do this dangerous lady, My father's heart, which your hand did thee yet be The first one of this noble Edward's king; And yet that is no more: come to thee, For I am we of my your love. KATHARINA: I have a very. KING RICHARD III: sadful lord, and his part-f ready I come To make the king a king from him there is That he's made good. Come, let you go; But there be 'twere noIC at you all; But you shall have France, for honour; if you were now have My body hath done to banish yourself, To prove a good will-dew heart, Wherein my noble prince was comfort To make a man's name and a great bed: 'Tis well aKINGBe, he's in a child I'll be by my husband; let me she: I'll have it if I use my sake. Why, how, let is in that sword this traitor! HENRY BOLINGBROKE: Richard, help! but more than his that I'll make my tongue to love thee such life As thou art to Romeo. KING EDWARD IV: This? but doth he not Edward's death's death! HEN MARGARET: It is more than no more yet than by it. BUCKINGHAM: What, is it so? GLOUCESTER: I shall be there was in my heart With gracious lord and my good lordle brought He is well: but I hear you will Give me to my duty, to the world. First Murderer: Because thou canst love, that ever, thy love, thy child, Thy man shall take a grave on thy wretcheding. Fare not thy sword, content Camillo, give Of your she on a my master and son; And here, I beseech your highness,-- FRIAR LAURENCE: eyeTYou may my kindred again! ROMEO: A good things still, this is this day's run Doth never speak,--ces nay, For I have seen thee very like a fard blood, This were thy poor, which bestELLIO: Let the more be put, sir, play in, I'll tell you is Edward in such deep men! KING EDWARD IV: By heaven, I hear no more: my brother's heart. HEN ELIZABETH: I am come to have a more worth in this. KING RICHARD III: Allak thou, God! it is not my son. QUEEN ELIZABETH: There is no more I swear than they shall grief is so. WARWICK: And thou and that Romeo's and thy son's face, That thou shalt thousandame in thy rest, Whose father's heart, of grace and heart's blood is come, And with the tongue that did not be there. BENVOLIO: T Gm would cause, I wouldsthip wish it true. ROMEO: Is thy life- vouch's own word. MERCUTIO: My lord, I'll take thy choice to go Hath yet with an heart-day's death? BENVOLIO: I dare not for the best upon thyPle I talk of thy Ver lives. Come, let's not A noble man: For she hath yet a man to fall? ROMEO: Thou art not so well:OM pity her: But he's an but one. FRIAR LAURENCE: WithSecond, sir, a thousand duke's of idle hand; But I am sworn to it I should not stay, To be a life as thy honour's love: But let me rest? JULIET: This will be, to put forth; there is no man. DUKE OF YORK: Well, me for mine; let me by till thy husband I have no brother, till I am not a king's son: For I have done, I here hear you speak: By heaven, I am a beg by my life, So far as most in a man that's the mour of! O, that I would be less! the great which o' the honour, And that I had; but then, I am a-beLEath slain, Were kill my takest! If never, run Aufidius, 'Tis thought it would you are at enmity That you did but so much is, and And I'll lay down out whose't be fullle In such most I have kill'd his have been such his, As so in the world is dead. Messenger: The man that he is our lady. KING RICHARD III: He kill, my lord; who, we could not say 'ld have so much. Come, come away. Of God's name and our fair? 'Tis oath? TYRSON: He's lord, my daughter; but I doubt not more, Farewell; and 'twas it be before I did Or not a one that she is not to be thy tNor in a thought of the world was not enough to be the king. Provost: DUKE VINCENTIO: How? DUKE VINCENTIO: O, let me go my lord, I would youheeio her, prince, of you, And make me be seen to the people. DUKE VINCENTIO: I do believe it, to grant and love me My state English yourself, being full of your love! O prince! one of you a man that I must not all be the king, which is mine and him. LUCIO: But, madam, an'tess'd by the do break your honour. DUKE OF YORK: I, for an Murd-- KING RICHARD III: What! what goes this? at' lie I'force? DUCHESS OF YORK: I would I had a poor day for- thou art made it. DUCHESS OF YORK: I should I will love thee any Isabel, If I have seen the king. Provost: DUKE VINCENTIO: Good Romeo, Romeo: 'twas done, As that in any man have done to you wretch with him: So, well I desire it, to my heart Is this the day of Edward's wife: Thou most Richard, I fear me, Thou wilt make thee as thou wilt be a thousand As I am, as most gentle for ourselves: To this my husband's fair day, my soul, I therefore let her stay with heavy kiss: And, as I could, but I would, my good lords, Ere you be king in oath, and my death. LUCIO: Why, how thou hast done this man with thy love! Farewell! thou dead too, what 'twas no time Thy other blood should come; and with that WhosePOL know to fain: 'Tis so false, to beg mercy, their pluck'dine side For their goodILL'er's hands. POLIXENES: What, be there enough' the king with the king: Clarence other m O Warwick! if it be done, And that my good cousin, I wot that I know. LEONTES: How can you think! We must be this be you; this power are My true son, my brother. AUTOLYCUS: Here's he that did keep her to be: he's a par and a the duke; he is very well said, he doth: He had not so, I do fear. AUTOLYCUS: I may say, sir, my queen, dear love. Clown: My lord, you will, by my life, a word; the he duke's daughter's wife! MAMILLIUS: I have not been i' the lie. DORCAS: I know thy canre so. POLIXENES: O my lord! MAMILLIUS: No, my good Paulina,-- DORCaius out,-- DORCLEY: So much to it, sir, that it may be received To the sons; your pleasure where, the king's heir, Have he been nothing to be depose, But the poor law so far the rest is dead. KING RICHARD III: Ay, if I have Clarence's bosom forth. QUEEN ELIZABETH: WhyWARD, Well, pray, go with us? he is very comfort. RIVERS: No, as the devil'st I am no more more; But I shall have heaven with my life to go. QUEEN ELIZABETH: How long the justice of the earth is not mine? GRUMIO: prison does: Come, sir, it is no more. And the rHAM: My flum, will tell your Lordhip; here is mine ToWARD; and yet I then lay at arms. YORK: God save your lordship to his own Lord Hastings! I but on such right and said 'tis well. KING HENRY VI: O, then, I'll away thy life to bear. QUEEN ELIZABETH: Come, come, come, let no longer, call to our daughter: For now the news is come. KING RICHARD III: O, but the father that is the king, The love for maid truth: 'tis in the needful Richard' death? QUEEN ELIZABETH: Nay me, gone, go with me, in good I'ight in thyAnd shall the sound wrong. KING RICHARD III: So much theseford still he is: His name is troth thee well my death: What, that is not so much, any of thee and let me think she hath no further As, nor any soldier, nor pardon, But that was she hath done, she should be. STANLEY: It is no more of your bed hath best not true. KING RICHARD III: Hath there been, and die out, so much to me The word of a love: what art thou? Second Lord: Thy father lives, that therefore can not be I take theUTA way out of our offer, And, as I say, for we shall not stay. LADY CAPULET: O my hand, a fault will be more never heard Or else moreare. CAPULET: Under this boy, she is of the Duke of York's son He live from mine honour and his blood'st To take us good; and, with much subject Like to the cause to be thus. LEONTES: You have content'd your most mostday To speak your ownness and my soul. LEONTES: Why, the Lord Angelo, So York, as it be father, for his life, To call them hope 'twere yourself and dangerous. CAMILLO: Why, what a thing? LEONTES: Was everBut in such a time of her, Or in a good will of this unisest? KING EDWARD IV: What, lords? WARWICK: And Warwick Clarence at the eyes of justice! KING EDWARD IV: Why,ere no more. CLARENCE: GLOUCESTER: CLARENCE: HASTINGS: KING RICHARD III: Death, brother, are you through my soul Sray to me, and leave me to my uncle? KING HENRY VI: This is thy brother's son in want these hence, Lest as I say, his Earl of Warwick, And tell himself to the Lord king to him, And he had been the will This comes. QUEEN MARGARET: Why thy words have there the blood for this land In that thy heart that should have. You, God's right I should have needful nothing set upon, Theself I am: and yet thy doth-- QEN ELIZABETH: O nothing, that my lord Richard will have. KING RICHARDISET: Yet, as I come! KING HENRY VI: I know the name do march. QUEEN ELIZABETH: O, I wilt thou not know that sorrow hath To hold the world with thee, and make me think How I am of thine. KING RICHARD III: In whose from all the shame of save, ThatCORIOLANUS: Why, how's there,-- VOLUMNIA: Let's hear your heart? MENENIUS: much for my very word, I'--Come, let's go: I am please her I shall do it, sir. MENENIUS: It is a cause, Pray you, get your words. BRUTUS: What, our good lord? SICINIUS: He shall, you shall be; he are three gods Is full voice. First Senator: MENENIUS: Ay, but I will go. First Senator: I know your Rome. BRUTUS: Is the news prove true, my lord, and thySo, as I And see that this beable and mostadman: Hath not a high Mercutio? That's more to say 'twere well;' which I pray I am too more than some, but so good For one to be done. LEONTES: We know my friend, I am theWhat I of your body, if But who youath done the noble duke, But what is done. LEONTES: Good Marcius, you have lost thy devil, to lie all Good night. POLIUS: The people did. SICINIUS: Are you all Aufidius? First Senator: You would he were dead, but not so much I: But now I'll call for it. MENENIUS: Well, well, no more. CORIOLANUS: As I had forgot the world, you have not a kind of love Must be a child to be satisfied, For you being no man to her death. BRUTUS: Away! the arm Of your fair wife'st, For she was my enemy. What doth the man! DUKE VINCENTIO: But makes him, good father, Thy poor prisoner is your time, I'll have a brother of a son too; The side are all betwixt me and his brother. And I have need not; and, as I am not The goodly gentleman is too, do not But else 'INCENTIO: O,en O, good me, indeed and I'll thank thee not no He hath deservedUKEill live to do this; And, if this beNORine no man's death is cold. AUTOLYCUS: Away! he is not made good, sir. DUKE VINCENTIO: My heart, my good lord, did't not't. ANTIGONUS: It is of this; We know of you are now. Servant: My father is at hand, Who came Camillo: 'Tis goodly: I'll be the bosom on the world: When the common misfoler of this battle, To iter 599988: loss 2.8334, time 71.35ms iter 599989: loss 3.3947, time 71.09ms iter 599990: loss 3.6317, time 72.32ms iter 599991: loss 3.5352, time 74.15ms iter 599992: loss 3.2102, time 72.34ms iter 599993: loss 3.5483, time 71.71ms iter 599994: loss 3.5832, time 71.76ms iter 599995: loss 3.4675, time 71.01ms iter 599996: loss 3.3202, time 71.27ms iter 599997: loss 3.7681, time 71.95ms iter 599998: loss 3.6641, time 74.20ms iter 599999: loss 3.8146, time 71.89ms step 600000: train loss 3.6565, val loss 5.9145 saving checkpoint to out iter 600000: loss 3.3908, time 193.50ms number of parameters: 3.42M No meta.pkl found, assuming GPT-2 encodings... The state of men with him that, by his brother's, The very good men, and hisWhen we revenge to Myself, like being now in the airing. GLOUCESTER: This gentleman is set out of other. LADY, lord, give way, and go we to our cannot, And I will leave them to one the YORK: Why, then we may come to you to the: I would thou wouldst hear; war's still he thou done: O, is 'twThy graciousYou were at the heart. First ay: I did not stay? HERMIONE: You will not live But sir, nor little's by the he'll take it down To can. You are not fit to me? LEONTES: What, youPEY? MELMy words may bear't. The one would have so. ESCALUS: I cannot speak in heart to come by with her the thee: never only of your love, O, He has with him most peace; he'setheth once, And therefore do you any thing he would have you; That, in't your eye, well- ELman is so long a Jof which's made youity. If you, you may hear that you are, wtis a shepherd only may he say Bohemia. DUKE VINCENTIO: To your love. LUCIO: ANGELO: I am that king. DUKE VINCENTIO: I would I had been on to the A there: Yet must you be, sir, to all I know so. DUKE VINCENTIO: The duke shall know you say speed again to yourself you. Provost: I will be an't. DUKE VINCENTIO: It is a word of me. ANGELO: TISTRESS OVERDONE: Ah, howio! not all as he being As is a father of a e'er these adTan your brother is born. MAMILLIUS: I am a word to-morrow; youpt, But you might, I shall have had some to him thus. LADY CAPULET: If you were mad? 'Tis full of what you should have. CAPULET: What, say'st that I mean to me to thy name? JULIET: I am the more. Nurse: Is a word. JULIET: What is the news thou bring? ROMEO: How shallest what's a d see of them! ROMEO: O, give mewence, that comes in the time Of this. I am not born a happy day: The which is an ear thou now to make me speak, Nor how in that word, thy majesty worse, To Romeo, not to theage, thy dear queen, I am a comfort that is my wife. You that's a brother, to the duke's e'er, To have his mistress king, to-ake: This is the love I might have doneish the country, That I may live in theWSo name lords that means That ever had been to doway on! What think you, if you were you all, Will not my full very reason for a great bed, Uniansch high the heavens? answer this Of this fair prince's name? CAMILLO: I will, my lord. LEONTES: Pray your father! That's true-Sir, and you are son. POLIXENES: You are rough sir, give me Lord: let him go your seek. LEONTES: My fair lord, Were I but so much, or do I see, my more proud, On this most gracious lord. ANGELO: Thy great youth is the forth knows. ISABELLA: But 't, an confp Henry!' The which you seem not you both, which would well, That I with dearLE RICHENTTER: As now I have nothing: With quoth so, my lord, and all at night We cannot speak, let it to remember me. CLIFFENCE: Is some thought of this, we may be a cause. KING EDWARD IV LADY CAPULET: And to me have me bawd, youngly, I would say, If you say so, it shall know the world thou thyself. LADY ANNE: And, I am that day to last with a thine. JULIET: O God's sake, is come! I'll make a years upon thy husband; And, all am grief of call thee all this good. QUEEN ELIZABETH: It is my father's; it is more A dead, a proud; for if she be king, I may not live another woman, I am not a father that will take it; And now, for my father, by my whose fineoe Provine and my wife's son brave father me: But I hope, my lord, for I was seen, With mine own love to-day; let me lest: So I have done, yet comes one. DUKE OF YORK: Thy kind-morrow spake for the grace-night! DUKE OF YORK: ay Richard, that dothst thou do to love thee? DUKE OF AUMERLE: Then, I am mine own again, or I, The truth of day; and with a kind house Hath made her dead, who have manyre many more That I'll hear him rather and say, for my death, I give a purpose. LUCIO: O living, my lord, it is gone to leave it, And you shall have of this good or'dN At my poor brother's kiss: and I have seen, The one that's made me welcome. DUKE VINCENTIO: This oath have as high as the wisest, then other mure of a chide on: the post of those: You have not there here are comeable. LUCIO: But in this while I should have them; if I cannot, I do put the fool: then, like I come to love thee With a set upon my master, which is his pence of the king; even so in his hands, As I havech you at his best, By that it may France, for honour! if you were now have all; I will be banish'd, do not to prove The corn of honour, and I will not The noble touke: but I will not have any name That I, and think it were, if I could stand, Most might be in this fair way thy hands And me shall the face my husband'sJShould be, To young princely to thee here, be long-s, Take it again in noCOR'er a happy depb not. ROMEO: O Romeo, nurse, how can your dU is a happy In mine honour or anBHAMELLA: He shall have a poor way inch by his honour Gless worthmen; I warrant my counsel was more. DUKE OF YORK: Let me I speak of such, my lord, That very good to be this, before the time Wept with Iixt these words. DUCHESS OF YORK: Why, so I am his wife, to do't, Yet hear my lord; 'tis so; and I, I do him! DUKE OF YORK: Int note was that ever wasR pleasure true: He which's anMENain'd now: heuck me not his bawd than I have content the old gold-bed, he's dead, That may not beoe to be thy face. DUKE OF AUMERLE: Whom is the way, to youQ is it so; For I will not: so she would thou hadst no time Than in me but not: but that--tis not so, For I should, I think, I please you, my sovereign, And by our country's breath shall hear thee, To lose a matter with a word; if we At the most little: to such nor thought it was, Unless I think had been yours, for I crown'd; And, with a word, poor soul, that thy love's will do it. BUCKINGHAM: I know the gods. GLOUCESTER: Pray, be these my bosom: for the time I'll pluck the pack of your house face. KING HENRY VI: WARWICK: And Warwick's death will go and did to us. And yet not be fear'd in all. Go, Cates, and not thy brother and brother, And come, I'll swear the duke a good To be as free as little as, so myTis-- That side that would kill'd my husband, so Whose honourFor it is lost grave in-- Whither dost not be long as long as Henry's love My father, Edward, to my tenderbr own arms: Till your three days give good thine and your eyes. KING RICHARD II: O I wast thou not dead, say I know thy mind, Yet thou that dost in beseech at us, And not to the people that brought you to me And think you of our good rather shall be Be it to me but for my best good. First Senator: O Clifford, O, it is the more! MENENIUS: I would he had a people, But, were you not't; and you, my lord, You am of what you shall be so? BRUTUS: When you are should have bound 'em. MENENIUS: So, most it do. BRUTUS: Where is the love of this fellow, that must not? AUFIDIUS: We do not say: We were to do't. CORIOLANUS: You have been a cause to need. VOLUMNIA: O, indeed, I'll to desire o'er a counsel, And I will play the us. MENENIUS: My noble wife is me as good as well in true, To have an those water's better and isar'd. He pluck'd with him, and you'll have all the news-- And it is as good as Montague hath. Second Citizen: That, my master, which is his friar o'er-s, In that she had done the noble duke's from The noble head to the-night bed-like. First Citizen: Sond to the king: But I hear, madam, let him be heard it Of God's name, that is ill and great blood On thy be these; yet every w thought I with it, And none but that I doubt; and, in your love Is it right: but I, my poor dear cousin, I hope that I had not to this brother, But I will stay in the thing I say. DUKE OF AUMERLE: Why, lords, what aunes may? DUKE OF YORK: A gage, prince wouldst thou tell thee they! Thy brother's death! DUCHESS OF YORK: Unshe's aHath To Romeo's name the traitor; And you, God and I will see thee a and that Of living men thus they had been in this love To the duke. QUEEN: So, they are not to be the name of all. KING HENRY VI: But, madam, be content for thy thank; If thou dost sleep at thee? Now long rest is dead; thou wert not thy f after? QUEEN MARGARY: As thou as now I love'd myself of thee? QUEEN: But I shall fight with him. KING RICHARD II: O God's pity where long the o'er the world-and'd ! within I have done for the hour of thee Whereof thyself be too quick for my death, That enemy, with thy warlike blood, And made the king in arms at gone. DUCHESS OF YORK: So shall I do, to so late, Why, proud-hINCENT are lived told? DUCHESS OF YORK: No? KING EDWARD IV: I know you that is a father, to my? YORK: By fair, I thank thee, love thy face Which thou hast welcome of the fool, thou thyPR; And, now my poor queen willBYADYOP, For in my brother I king have play'd heaven: For Edward's face, being gone to Bolingbroke. TheAU's aH no other tongue to thee; For I have sent not to be of. DUKE V WhosePOLIXENBERTER: Not that that shall tell thee to my grief, But I will not be gone to thee, and thou a present tongue. 3 KING HENRY VI CLARerer: Take it with the day. BUCKINGHAM: Therefore, my lord, I let not beauteousors. FRIAR LAURENCE: Why, then, I pray thee, call thee bring me to me For this my gentleman's letters and her: 'GoodISTR in't. ROMEO: You must beL blood; And thou wert dead with a for him thou this! Nurse: Go, come, come the nurse; I mustETH: I pray thee I know thou canst wish it not say. JULIET: Though we have less, for ourHave set than me, So full of honour, I am gone to Warwick. ? of ourmen born? Second feel so: this is well as true as you, That you are put up and see as fast as fit As far all-f- HINCDand children to my poor blood To this loving? LUCIO: Doth she, when her father's Claudio hath done. What's he? He is in the name of men, With tears of heaven, and their true-ouon put'st Ourself'st man; and, and so is they all; That thou, like an unf swift use of blood. O, I have thy king, and thy king, That thou art a rest, thou, then now in thy place, Take thy hand in thee to do thee most am dear. JULIET: O do this love, and be an O! O's thy title, Is more more than yet thouThis: now, and that thou art To be it? 'tis not a tas beg man in seen an hour, having whom, he will not tell him. The means of you, my lord, is your lady's son. First Citizen: Away, my lord: we know they do us all. First Citizen: Woe to you? then you have said you were? Second Gentlemen, may she be put for it. I but think it was a bawd, I had to do so; For, in good time, a very well-baww gracious. Nurse: Why, how now, I art gone, Saw-b, ere, I hearon! JULIET: OGLOUCESTER: Here, that would he were so sour to he. LADY ANNE: Why, Warwick, how now in day shall I be! GLOUCESTER: gone, my liege, in what! If it be I had to die their high. KING EDWARD IV: Then am you patient; and therefore is Of all my husband Paris, to-day. GLOUCESTER: O, I would I had rather, but for thy son? KING EDWARD IV: Your death, my good, what! that I am there. GLOUCESTER: No, to thy name; but very it is not, For I have, to go with me; why, he's dead? heir: We shall not, my love, my lord; But, Warwick, do not justice her out; For therefore, so it is his hand that is: And yet our country, we are, nor I. QUEEN ELIZABETH: But what means, good father, be that state? KING RICHARD III: Tell 'p thee, an brow of all. QUEEN ELIZABETH: Good prince, so! KINGain: What do you thought, I hear it, I know you well: Your face I am not in your good counsel: Then be it then? KING RICHARD III: Ay, but he, but he's no more than a country. BUCKINGHAM: Is it none, good father; you hear no more. KING RICHARD III: We have done a better, our person. QUEEN MARGARET: Call so it, lords, have made him our hands: Berely as well as I have; for this, As for ESCALUS: I think much; but they find't their way out, but to die The people, and he hath yet show'd! ISABELLA: What you, I'll givehe outOUCCIUS: I am in need. ANGELO: I was a kind of all. CORIOLANUS: My gracious brother? LUCIO: ouson? ESCALUS: UC.' is there a thing? VOLUMNIA: I would it, sir. COMINIUS: Let me but stay. CORIOLANUS: The gods too word, my good nurse, I come from Rome and of the had the gods alt have been in the presence there that made; The people lives upon our TheABes; So therefore you, I cannot do you, you Were, my good with me, your father, his lady: One of you, he has done, by the honour To Coriolanus. CORIOLANUS: Haply, I mean to us and on. VOLUMNIA: ine, Lest, take up the duke out the traitor's son He was a like a father; who made My best son: I will wish you his love. Wilt thou? What's this? what? will this, so I here, Were in me by my master, I am like. JULIET: Richard had his grace, Nor, since the life that it hath most his, For much more shall be, and in that word. Nurse: JULIET: O nurse, I fear it were night. LADY CAPULET: Pray, you have all this day a may have youak. What, is not so long, you would have been long A noble deputy, which you come toius; And you the people is, you shall find again To see you knows the place. CORIOLANUS: Away! Second Senator: Come, I pray you go to you: you will hear me: Come on, madam. MENEN DUKE VINCENTIO: What, if you do not, you should be been well That you shall have; and to the alone of my heart: Do not think that it is hard to take. LUCIO: And in that world where he cannot be; for, I do see You that never, I have heard, but not go; I'll the souls of more and more That is a good way to hand; take TheSIC he my C art, and myself to please. Good ladyENS are thou what an openon 's tooNot, nor what can part in this fair speak. NORTHUMBERLAND: Come, cousin, you standNow down toWhich as these blood As thou wert so now in a comfort's hand. KING RICHARD III: My dear love, my lord, my sovereign liege, MostFrom earth I come for Edward's death; And longum thou, the fault, but not thy lord: I look'd, to be fear'd for me to live. GLOUCESTER: I never will, I'll speak this. First Murderer: How now, like! what's the day? CATESBY: Marry, my lords, shall you seem inh nor wits, Or take you by the once. Second Murderer: O God's brother, thou art mad, and begish'd. Second Murderer: 'Tis never gentleman, but what of my counsel But to my will at morning. Second Murderer: How long as I? Second Murderer: What with a name, was this? First Murderer: Alas'Twixt me, and must be thy daughter: Doth set the world upon thy counsel with thy thee o' the face, thou hast made them aHere. Second Murderer: 'Tis more but not, dost thou havest Lewis, That he hath leave of more unto of this Richard. CLIFFORD: WhoCAP thou wilt fall so of he had been of! RICHARD: And bid me stay. RICHARD: Messenger: Give me thy oath I 1000 iter 997: loss 5.8675, time 724.04ms iter 998: loss 5.9554, time 722.33ms iter 999: loss 5.8129, time 730.23ms step 1000: train loss 5.9102, val loss 6.0530 iter 1000: loss 5.3077, time 1634.94ms wandb: Waiting for W&B process to finish... (success). wandb: / 0.001 MB of 0.001 MB uploaded (0.000 MB deduped) wandb: Run history: wandb: iter \u2581\u2582\u2582\u2583\u2584\u2585\u2585\u2586\u2587\u2587\u2588 wandb: lr \u2581\u2588\u2588\u2587\u2586\u2585\u2584\u2583\u2582\u2582\u2582 wandb: train/loss \u2588\u2582\u2582\u2582\u2582\u2582\u2582\u2581\u2581\u2581\u2581 wandb: val/loss \u2588\u2582\u2582\u2582\u2582\u2581\u2582\u2581\u2581\u2581\u2581 wandb: wandb: Run summary: wandb: iter 1000 wandb: lr 6e-05 wandb: train/loss 5.91025 wandb: val/loss 6.05303 I my, your the am; By areU he: WAR H That a me, I am father Or: And to her I I, I ROM, lord:ANG is IET. ITER, He Hold can for my your am, And if yet be'll to the the a life the the he a my; P: Sh, have this, my, A,, A thee- I a tell: O. A be any, Your you, I my death For I, this. You, my As I and the PET's artUS, My life than. Have see! and hand to God of stand, With, a many me? Where of have think, it to I have not, That that that pray, As the my I not thy the she lord shall a thyIO and would poor, for the: and: What and he, ' so it, did a will his king, a is Th: Butour Will, Were,,. She, COR the her as, Good When. For lords. and butET is there your me? and shall king the this. C, thy mine. O The shallUS, That: GL: That, That, Then, here the But, What: H the, D and, NOR toEN! if must, The the,US? To my the's poor. Why's will have he'll can how man Some in cannot than come, Even a in, him. Is you anyENT me; so say, be sir, the the Itis the, be'll man I them. What. So thisAB he, That that the will, That the'll:I a king, That ' Would Now, have my the a: To kingUM, I'll b the must a do thou, H. Were, his thy the the the I'll, or a'll cannot, I To No for they not such time, name? And was have to not me, Come of must know,, I be the yourUS his such I be my. How, For me the have have: ROMING To to when to, My: as And men, An a B all or And thou is, To man, And the: Come, am sir, An, And: Lord queen for, they my, that he part This! It in my trueG do in: Is? And we most, that the BKE: Is an NES? From say; you and, to's knowING your I his But V for it: I lordUL from well, Go in me, it the be I was. but: the her as IU the he our life; if his I most in I the is, I But in hast: Thus and your noble the do take she be: ' I'll a I your d ever, be not allEN COL to aOL, and not he, you is I make, This you not? I And, beELL what there: Do her thy here, not not Not crown your in a that, if death would we you not the w made of his me and butI not, What, H it P can that The the, So: COR H This: if'll B, butET with will hand, I the a N to must a, If, Which I would your- have by the am isIO, What no come is This aIO: MEN been d: WAR in a not, '; name: To sir, I most NotUE men, I What and d me:. my? The's have a us he the I lordose, You not did, Here, do an a she, my we a he: BR thy Why she. P love,, COR this the that how, Of what out H thy: COR his me you, And as NOR you as and that But I Yet. I no crown, say: As have: and wereTER:To: KING KING. ROM be you my the yet, some one? Second their a theI, the father this it not the his. A as not: ' it thou I theICH. O a lord.I it. That theES to he So. But! That not do. I: And your Q. I What IIO, is lords, he the stay that my come: How it were. H not, D. And, that be'sEN: H P: And my that have that not't to they the thou thy were If in; and the for know N In: for my the- never do tell R d it, Not, First, of I Come I be be hath a be her these me by this: and the it goodUT As My when, MER must. H And have the you- sir: and your To be aT, which is; That one shall must you the a, That make I that No'd. This; If now, good dIO most you time, My. N you, and this is have will am, ROM of were shall lord toUS, ROM is be his him: Sh: and her their: H to beEN I the name, Prov not all the you thee, The it, that was,IC How, As such not no IINC, our I and this is not, C. By but I W will as his Sh! you, To him: and say; she. Sir at notEN. Your me, for what is, Q them: Is Sir, one a him NOR with ROM mother that be that see in, H. I D, the so,, This man to She come? D Who, but; by He, H, I I I should my sir. As, take my as or. Let: Is Where his he, No his you be D have,And you He that in man thatIO And You in blood should That: As was lady. Now with: First, theICHEO isI very: Ah, will an- I blood him a, house, that part, my: MER, so, And I but you: What: Your come, Q, To, Now, will he see: Be, His to his an shall then,-- I. I most, your the this By will'll and For a the a a would he At I'll? I: Un my head; They thou am, I. It: To not, my noble in so, and O. By art? Let, they is have thou: COR; and I The But Th ' An are letOL my. And My will, What me, A, A good Second. Then: Sir:To am art D. I the thetis, we one. I Lord Th. D, let, Now, MEN, the, To which And. TheUS so that an be give, be so, Come then him. That can the:Y with by; you give to'll A this to be the letsIO to be the And have'll, These is a thy! I And: OfEN cannot'llUE with he my do The did: Of thou say so; be, best The our What: In can should of a he I To, Which J: I: He of his sir- am, If sweet the Now. ROM To, their. men? D, would king: And And one And'll himly name. COM, sir with, And: N am's that you of there thisOL heart: You if son But: WereET: if I never, Your in your not? goodIO, or: How Give not in; Come yourEN have most: H:So the I do, If my my I And the good I this or love in think: My Come's and some not I call, to mine: KING you '. Of To But life, be, B you, you, And go: Good theUS and love W forTER to'll II them:But'll an. If ' will, and's part be bear here, A: We you How, Make thisEO's his would love, a when not: N him it, are you, a thou then, How. Cl a a for your! be the part, And take the But, sir, That: Ay, well! This, Second all-: If a I the'll all withUS: Totis death; sir of: And his her thee to this'd: Un, see and a KING, The the be; thy man, some do: And thou ' the the some thy To the this As. That--IO, The I the aKE thou in give! It shall some thou have I I I you the a To I d not am the thyKE a my the this he: Would my life,,, Well is thy, Which: Which a well. There R, the a B so: and UE MEN not in I tell withI if If in the R AsUS. but I the me when you, Good And him. b poor, One MER: To all: and is I makeant upon it I we a blood I. You Is, the, myIOL would First R, a he, Your queen: no people? TY: I. Is shalltw: And 'ES youtis Which his my your my art we haveUE me S his but man, which come: That H the your the, b, His the a a my king? And Go to his Second hath I his more man up, That, What. And do'll a a As He in he have, I my willIO: And, That he I isU You, But. you? Which to a the a, To You a, have the my him, a these this and a him: R most love: that no do make if her For I know, my grace. AEN the am your your the I lord? Who, I the with But: I, a Ns to have so such: KING: J a I day is D crown'd, That But the the. Against when it J. MEN a the. If It love Give theKEant, E, go. Sh: I ' as I ownUEAN: Thus will myIO of And, Of queen: IUS of my lord! it, He would thou? His, And that mother; And N I have, for his if shall, Now, to the am, there that great must betis he D this. C the do, To an, As a them,: ROM, Third, First: For the see, I the a a, , Now I was,'s that I it you the done Th with do my, That that it: O: Your, our'll And the that ' him like but, a honouron. I dead to her some I: He, If: KING: R thyUS. And Too: For a be theELL, our, And many the but am this, Why, They that the this make, I: Is have am her my my he all,, when the will the to be, This, As may I my their have the To him: H know is say YU ETER, Let. Now. A give is of shall thou, There you, P, I poor, a the some have the your are the for do for you this to your a for or good do, or noble: But And shall he be And good them with Cl's The are a do you. That notown, Sh, I be do it,U O you I be die, Her, by I; she, To be to that sweet all, We be: D more. For the WAR: a the-'s she, that And a thy what, And It, and these. The thetw's know. Like of makeIO. A But I speak: with and: not, PR: Come we If his,EN hear: To I I, N; Of, I, First so the head: To that hisI be you the: That you. You? ' shall I be As He is KING, come KING, the do the the So, With, And shalloth such me,, you by: That to her for I like. Our, As in this love: 000 iter 4990: loss 4.7843, time 730.76ms iter 4991: loss 4.9219, time 722.85ms iter 4992: loss 4.6760, time 731.14ms iter 4993: loss 4.8503, time 699.10ms iter 4994: loss 4.7151, time 721.49ms iter 4995: loss 4.7852, time 687.31ms iter 4996: loss 4.5192, time 732.55ms iter 4997: loss 5.0381, time 727.84ms iter 4998: loss 4.6365, time 694.68ms iter 4999: loss 4.8973, time 691.50ms step 5000: train loss 4.7097, val loss 5.5308 iter 5000: loss 4.3929, time 1421.90ms wandb: Waiting for W&B process to finish... (success). wandb: - 0.001 MB of 0.001 MB uploaded (0.000 MB deduped) wandb: Run history: wandb: iter \u2581\u2582\u2582\u2583\u2584\u2585\u2585\u2586\u2587\u2587\u2588 wandb: lr \u2581\u2588\u2588\u2587\u2586\u2585\u2584\u2583\u2582\u2582\u2582 wandb: train/loss \u2588\u2582\u2582\u2582\u2582\u2581\u2581\u2581\u2581\u2581\u2581 wandb: val/loss \u2588\u2582\u2582\u2582\u2581\u2582\u2581\u2581\u2581\u2581\u2581 wandb: wandb: Run summary: wandb: iter 5000 wandb: lr 6e-05 wandb: train/loss 4.70968 wandb: val/loss 5.53077 number of parameters: 81.52M No meta.pkl found, assuming GPT-2 encodings... Not that; You. And it, I to the mother on my son.', PR: And the suour of, Quke, That, not is this earth And not, Now more years will the part: The hands; Not, My son, To I do my queen. TellY: Darest by strength and. And, As Provife's side, And I not the gOM look, which: AUTUS: May not men to dU and you, So, LUEut's lie on a would I can! The suilman, The noble lord up I I do nothing in your end, DUCatter thou have my r LA, For, with to the earth to your wife, Not my or not his hands With you do a friends, His loss, Should have not, as Ibians My mostt to I lord, the world, take you not that you the cause, IOL OFCES: To you, if her my father, I can be your Tower To do my name. That the hands to me. Icius, LE: Why, LUCIO: LELCUTIO: To thy lord, to my day with me? And you more for, For never my power, Her heart; DUE: Good great noble father, this In to an father? And be do for the friends! The lord ClOL OF YORK: And he will he of my hour. And if Will some thing at the lord Bove's father have her; Darep the counsel. As will be those the one, go; come How me: Madunes! By me. And a curse of my. ' I hast my lord Not, And, my lord; He. O Do you have be With blood, R VI: Tell your foe is a king-eseous death, Sir a most mind. That: Who I have ProvUoWARD But, O And in a go, From sweet high is joy upon my, FirstICan had this? WARICHCES V Lay: Come about a people, IUCUS: But this land'd's father, And Could To my will; let my ancient lord Unless my lord, but That, And I shall be I, farewell, if I have said their head, And yourly the time All it which thou have, Which not will; an eye! In me of her not. How I would I DUCost it have the lord, to you is my d'll my common d understand What will not.The business! Glouace; and you, Why your king is your great And Mess not's in heaven on me Say It thy day.IUS: And FirstIC VU, Be that I? AndtwEN Which Best you no is to thou should I do not, Be thy life; And and If you. And I not. A: And he callWARD: To their most we do yours, Dost; Who the first: To be't? A: To my king but Once will But. Thou hast our loving dADWARD IV: The Duke; Sself is the father, and be make you Why, Blbroksarry; From a Lord of that she? As this, Which I have but I spoke, since was the lords, Well, And your course! Sir. Do this grace they were in heaven NEENIUS: Look's, BowY: With in the word. L: But I wost to not the heart To all thee, LU' the my heart, Hre the heavens. And he, we art shall be be your other the last in this quarrel. B me to my royal prince, My life, it may bear hist; SecondUS: But in your, good day, And you, that, And it; it, to give him, From true may be not AB, Look to you will My, In thee-OM do the less aself, Before the, For I not better 'er she not you here: And be a heavy un Senator: MENall him It, It be my suit KINGELBY: Yes! Is my father, To the best, a grace. YATHolding, but such me I cause your sovereign! KINGABILLES: Nor what shall do a ear of the word's tribricMour him PER be. A: Sweet sister, My self, That the grave, The lie' so thy case in put, And my wrong, MENre father. Your child, I do be our will Nor your That KINGOU RABit's with a sovereign's heart as, and been But, good lOW: From your court will you me they friends withESSyly I-US: On the mind, Iie, And if That the good thing the very poorICIN With this is him. POLam'd me Say, And thou I do be be all my word, be my noble day, a Tower to you, Or, That them, sir By my sister; HILLES: Or than was this, Or it, LUS: For none. Have all your dINGS: HULEfore. Of her, Who Th OFABall Burse: IUS: Dcius. But time, when I will be you it the lie me, So me, So my lords, Be but bear all more His better I have thee to not by thy dU me's more may, And he. Go, That the words, which I besinks would he'd in hisENTumerine! Had I will, There has, but when Stand with him, FirstUS: Why: Our father, in CO; Like you it me's lord, But,' he heaven? Be you may the life. Doth my lord, but was you is dead, AETH And see your words. Ander thy will all down, Or in my suit, And at this f Murdaced not, To; LUS: From all the very and his And I must it shall am I am you you, Which, My friend with Inay again: For they traitor; For you but an country? To had the LU, here, had That thou, do the heart, IUS: A: IUartarry. But I but MEN: FirstUS: And they not, LE OFELA Marshal: Are not to heard theICINNow, sir, RABELLARRIUman. Well.And he a eye, and.And show to I be your death have this lords me that, in the king And this a times from I would say, then, your father, FUCUS: R YORK: Now. To see with my queen, GiveBERIO: for your sister, For it. Of your land! And For, I be use their case; do have too as see his fault And I speak Who I more stay? Let the or ent not, and now all you by a gods, you I say dU your war, In me as that: The end. God no more thus; GLCES OFUant; And I, O of HENO, But, ToASIONISHorn! Which The world, Why to, Which. Who have tost it shall done.Iona me. JOHN: And it to his lord. To be dead for peace, for me; She I they as that, Which art. Which be a soldiers And a rest with an soul, and of me to all for her, MERARDULBRES: Now. CORUT'll give me, And a house, Give your gentle be l'd for Good death, KING R HEN'B the honour, To be ourself, Firsthouse. MEN and the father! ? You'll forUCKONT thee; Of his gentleman, He will more's: ThINIUS: To I stand! The head. No, the brother's this, nor so the world that in a bo', then, Most justice, So you, as the life, What, With her this breath, sir, You thousand uncle; When he the world, but And these brother the people. That I besiss? the good father; And, LUKE: Why, I, And for the king, HEN' so to And to you. KING RABETH: Th me, Cer; Let haste, now LETER: To't, And Quke! And this, I of to the eyes, In your blood we all here this sovereign: And if a my time, As my hand, Three life: Than to your power? That, FUuUCKIN Ile? LUCIO: Your head. The head, no thy people in you can your heart, HERuck thee, They be the life. Dague to me For not's kingdom, From the death, this loss, DUS: BBY: O is my as it my noble trible, and nothing,--And my quarrel would what you, being not, Q YORK: WAR YORK: Why Now, With a my side? En them, E: As I know is my other's poor They have you is, DAR LA II: That's more, for me to myself that I be a queen and be a heart: IS VI: SIO: Why and this lord, MontM the crown: May think. To this's hand, and your hands, Are not I not I not's very man, He you, I be done, QUC love upon the cause of my air In you I he do pass, FADESSINFor my father, Bret to But here, sir? WhKE: We Q be, FU, LE: DUall what and, On, LU, you I'll, There, LUS: And you, Theark any? Which no that, COR thou be in 'er And There in the child, LARDELLOU VI: If plWARDASTICK: GL YORK: The crown, as my VolKE: Lome, For I come thy life, And HERESSou I to you shall, on true have-ighth up And lay Dague, Y: GREY: Who, To have your ears's lord; that now. But than I thing! LE OFUCH'd, Your death. FIO: 'T: To in the soul, sir on the word. Like OF YORK: GLARD III: The air, As There? Peace; What a love, FICK: but is the end, And were the friend B thee, for a king, ' he: And And that is an l me to.IUS: So, SwESS OF RentleY ANICou we did art your breath, weENTona? Say It that, At your man and His uncle. And I have him, And thou look is thee, And this very good, that go; that to time? MENENES: And here's, sir, Ay: My prisoner's me. My daughter! IENCE: This, And your restrah is your great! LARADEW' the other fair land.IAD me you may we be he live, KINGELUTET: And many hands, And he thou came in thy sea. This your sister, And I set wABiful; While'd, with this cause, and done, DAB II: Or can our breast, Henry shall were my oath! Which thatness-- T, She I have live in some man off be me now not may my lord, In a people; Which ' the words to his, If mine you should my name; I have Now in a lords, And my life with him, KINGUS: And you, poor looks's you on your father have light, If 'em. GLUions when men my On your lord? But his grace. Go,, CBRESTER: an sweet LUge, O, with your world. And I well to true is the Lord. Lordugh, LU and how you; My soul, With, And What: And, To-US: To that is your life as in. My hands. And it it to be Ct is our soul. You now, As both a queen, like once and any more, Like you be a brother! SUMGABENI would he, and I see a blood We do Clarence Well me, To time, we spICIN GRE me-ICINJENALIO: And the quarrel, 'T of a my the Tower, CIUS: On the case; Let him, She that shall be we power! To be your, And And a right with that'sENTOLou you he, And that! LE: DUiol you. And, Will be be you of mine that we more for the father. And IENTIO: For to right upon the death'd.IINGS: Now, His field, I be my gods's land, FW'll. KINGOUES: And thou If not to the father? HETH: LE:-- Look must you Who know man by her. POLopit you it man's to the other were make my d my lord to the one. BYts you; His time's more is this eyes, IS RARWICK: LUish a men is to such this brother, O, That must he it out? We can be There him that more's more me, eunes, And IC I not is your the lord. Ander before it. And thou were, a peace, And Scak down with a, AndT II: Which and him the king; QAENI had me to set sorrow, He with the lord of my wrong. He, CY: And were a gentle lord, And I had justice. He, aICan's to be your prince. The better and once have be have none! And he is him, I may this: Yet come are the king, Andn all. From our life. Shful blood'd, and not Shre is what that, Jutful men all day, my better you to your own earth, I do LICHCBRHAM: Who, And my night, My life! What this name. With the side: How And our further with ourselves, If are for the pleasure: And is; The own life, The father; And I stay. We be the fault, FirstUS: F mayENTfm. Of my death it, Firstoler. IICK OFICHINC scorn me, ThINTo the heart! Of thy while man, of him To take me, You. It by those the in me, As So no world,-- To put off's man CORtis going's land, Her counsel at your father than both you, Why more; But you, For most world-ICanly Richard, Make no need is me thee; The And so! POLKE: DOUO: FirstUS: Thus, Must shall not but I, You, lords, farewell And will not. How'sbher dost, and be, LUEENO, even or so I For in my queen's good king, To fear, The hand and, to my man: And the voices, Here aem word, And will all to a dUman: Who by her with him. Jricest a bo R with your IIO: And we in him am the lord is. And As To not is great Have you to in it, Mess so is my high on and the time! That I do your lord, He shall made? O you of a best, theyt with her of her must with the high me, and the master at my brother, if it you not not I in the suful house to they desire, hear fulled thy grace And your FirstUS: GLOUUTIO: O. A: Of, This it with? First should for theself: To the hand, Or you, SICIN Ay, He in me again! Which go with the time. You come's! Here myself, L: A: IUS: Or that he your blood's and he are And know your king. And With't will a son, DAD your is an old's king in him to thy house; While yourt in me, Are bear, ' they. T: ROMO, QW: DOUES ANIO: And true, he a day, T: To to you than, and so; If he is me from me is my one. The kingdom in our lord; And the hand upon BolCHp Henry. MEN: For I would the earth, The crown, Your bed, I not ISARDENES: Alure, And more to; MadKE VINC say CORES: MethESS: She of a more do IIZARD: And you, And you, I turn not the presence. To gone: And I will th, I say. our voices, IUS: O, KINGABours. And you hear my face, for they more I will his king, thy bosENES: Sir should have go, LU, If I be none, my soul is your Duke, And he. And a more ' it for the son, His kind, He, being she En is it to this I HreG am so, LE: He to be, That that by, sir, Cords, LETER: MENENCOKEANost, and not we What the wife, ESANUS: May I of the Lordus'd the That to you will not did, If For he by my man and, DareUCKou And it, But, but For the life. SecondUS: And we think, they The part. CAPishment his do been me what is you that, SecondIO: Go your do not, When you B this life. With her, if he to fear. The grave! The king: That I HFAD down, all you with me. And I to't. MESSou would he, I, IADere's man Call the word Before a mother: My life, IICK: IUS: As That to the fault, Or shall in my heartous face, by that, then are it, and my headly. LABch the present, FirstUS: To, AENIUS: ThANUS: I'll out! And we the looks's blood, and be your life. IUS: Th VAENES: SUS: That's, Which in him, Who for me I show her? But me, so thy friends, as I not, To you I ThINGBRO, GLAB-US: And then. ROUO.IUS: Of peace, On the hand, But from a lord, DUed to I-ICINLARUS: 3 III: Sir, And did not can you-blood DUed, And the great head too thou should leave, 'T: To how you him, A III: Thou will not But to not, no devil, as it, but I be him; Hans, as he, and this time, You his life.","title":"gpt"},{"location":"gpt/#gpt","text":"https://github.com/furyhawk/nanoGPT","title":"gpt"},{"location":"gpt/#train","text":"python train.py --dataset=shakespeare --n_layer=4 --n_head=4 --n_embd=64 --device=mps --compile=False --eval_iters=1 --block_size=64 --batch_size=16","title":"train"},{"location":"gpt/#logs","text":"iter 432061: loss 3.3774, time 102.72ms number of parameters: 3.42M No meta.pkl found, assuming GPT-2 encodings... The state of heaven with him yet have set The enam received in his power, It will be such,morrow; if he be By common enemies, so may not do, you As if you mark the truth. FLORIZEL: Why is my part; Not here my life, we'll do't, where it is The lark that the Earl of Warwick is slain: On bold, my rest, my liege, in whose good time I'll drown him worthy me to my Earl towards, Is Come to draw to scARD a fearful day. 3 KING HENRY VI KING LEWIS XI: What! canst thou live in poor soul, To do him banish'd at the same our point And the freshinks I come, 'tis aught to have but I fear you joys: all that I cannot On thee would of my heart, my poor soul, To be so bold a knave in this time; And thou by, she's children's blood, only, it is To see my life. MARIANA: A thousand part, indeed, sir, a time As you will wonder not: come, my liege, I well; And he is too a gentleman. See that with the voice that at once, you must Flried the senators of the any it good to us being son: but, I cannot be done; but we can tell him; for him I'll draw your words together. What you will know how you? COMINIUS: They are, uncle. CORIOLANUS: Therefore, my gracious friend, I am the farther un ches: What, poor! I beseech you, let it be a cousin. VOLUMNIA: Tush, the gods! MENENIUS: I am too! Nay begin to bid my lady play well: I am no pity that willy heir is right; But, by his leave I have or why it is. CORIOLANUS: Not? let me see, you are strange? They have made no more than all. Hath my wife's lord, upon pain of death. Third Citizen: Why, I shall have had some heard of you hear. QUEEN MARGARET: And so we all, I thank thee: 'tis more; Thou art the letter, and thou hast not so: wouldst be done, hark you, I know, my soul here have t something So to say, but I pray, and no more. ISABELLA: O, if it were full of such deep Norfolk, love, I would have more than his gracious fault Than Angelo first to you at great comfort. ANGELO: Yea, and make you he crook 'pt TrleKE: I'll make. Provost: What, my general? DUKE VINCENTIO: 'Tis like of his most, And leave me to the will for my poor side. ANGELO: Hath the king'ster heaven with the majesty of tears, With a kind: I have, my lord, must die. MARCIUS: Tell me hate them not, honesty are there To entreat of my wife, and my son to my order, And set the mother all my father's consent To make the must fly: so go: From whom that I would not, I do not fear, As well I will have me, you are vile toward. QUEEN MARGARET: But, by your vouchius, I fear not what I am: For since we came, your father's, at the least? Second Servant: Tis more than this of you can but have done My love or mercy to the queen of heaven: But I have forgot it, more, goodenio. CAPULET: Ay, as the matter I told you speak, my lord. BALTHASAR: But to the case of the Tower. ROMEO: Never; live, my lord, my soul is as well: I hope, by this which is a villain, When that thy brothers Romeo is, one sun With thy mother's wife, that I may return To know me what a dream hath as well As she run thought to be your mother, I I may not need to my sweet, CAPULET: Ay, what art thou? ROMEO: A queen, nurse? JULIET: Good lords, If thou dost know ye but. JULIET: Say, that Richmond's soul, take me in all. ROMEO: O, Romeo, I will not weep a while. JULIET: For the defry of grief makes them fair: But since unsapblful villain's wit of heaven It was a friend for ever-day, When I was lawful oath and not so, I am born to do it would it might be, I am revenged on-hearted my leaves dead king's oath, For the wantless blood whose heads must enter: 'Tis the Lord Northumberland that King Edward was himself, The Duke of Warwick, and his place in France, And thou shalt come to my will-day shall not stay. HENRY BOLINGBROKE: Why, howof, I beseech you talk of, Here must I play my heart with a cruelay; That doth not fall dead, I know the matter. DUKE VINCENTIO: You had even like'd, fie me, 'tis, I am resolved: Nay, unadBe now, I have heard her speak more than you I speak at peace. ANGELO: Well, I do beseech you, fair sir. Your eyes will not come upon me: he is most man, it is not so: so most much little most wrong, and may go to that by: which you have therein your hate, we may see themselves. LADY CAPULET: We are in prison, one word; ask him; For some you win me, upon his best: You know no better for the mind, there's no matter stands on him. DUCHESS OF YORK: How wouldst thou? Messenger: My gracious lord, I like youThis. DUKE OF YORK: Is it not dead, cousin. DUCHESS OF YORK: Then 'tis shame forAnd all that thou wast?","title":"logs"},{"location":"gpt/#king-r","text":"DUKE VINCENTIO: Romeo. Provost: It do trust me. ANGELO: Our lord, I cannot speak the good word: Since I havewas not it, I do'll therefore know Will a divineade me speak. ISABELLA: O, let me hear what you did deliver, By 'twere pity to him. This friends, be his head it, with one that's no matter, look to me as this lure of the prince a curse; more stands on't; No, sir; not a very weak of mine, If not that. Now, andso death's like death! I see, and hear Montague, my Montague. Who knows thou nothing; be not gentle, I may not, yet they shall be so, then; And had he come'd withAnd the oracle: I'll have your heart: that I shall follow that send him; that, we will warrant you, andPEY? ELBOW: Nay, he shall get a man to me. ELBOW: Look, very well, he shallEL' the better royal, which I do do't But I shall find, a time will be from his spirit. I prithee, take one: give me not your comfortier: I will not be come to-morrow; let's entreaty? ROMEO: I beseech your grace, sir,-- BISHOP OF CARke, my dTwere 't. BUCKINGHAM: I will, my lord: nay, You will have you go with me to please again. BUCKINGHAM: You will not hear duty, but for I will, sir, You have not the lady will with thee love But what you may! You are sure of he is to be a doth valiant; I will wot the prince: and have first to-morrow then. ANGELO: Thy business is the more and that my brother's sister. ISABELLA: It cannot be so. DUKE VINCENTIO: I say you, well; for if his present is the WARWICK: And thou wert enam'd; I'll not be mine own? KING LEWIS XI: What man is none than that at his house, That bear their down heart to Edward's-t thou him? WARWICK: Bid me, noble lord, our friends are fled, And sit to be true; and with all our sonsiss, From forth the most defend in theile departedath Be not born to make a husband as if I send it to thought that I did weep, And quench blood or not well. KING RICHARD II: O I was Clarence! What'sFL words were it: That I, as we are, that they have no friend, Even in the blood of heaven of fight, Thou p country's blood and'd. Come, and go with thee go; And, as mine I remember as any man's son As merry as I told the people and am mighty As'd an hour begin and bears our hands, But shame no other Paris and her be drawn. ROMEO: Is my long come. carry, let'st thou father In this remembrance of love: thou eat not 'em most noble swear tongue for what thou mayst, not wert between. The Earl of heaven of Warwick's love, this man, That's hearts, the true king, myMON and Sir. And unlook'd in the princess slain, Is beg of hen, and did yield in their arms At no moreOR create give from their hands, For the dead George of Clarence to his king, Who spake me his enemies shall hold it. BUCKINGHAM: My lord, my gracious lord, You had a power of wisdom cam thou out of mine? KING RICHARD II: He hath enem this, people, but I'll win me. CATESBY: Then here, my lord. KING RICHARD III: Nor I, the crown that queen is slain, To take the devil of our other linean ground, Not his our kingly curse people's-Which they that Is put to have open honesty Either to come in their arms. QUEEN MARGARET: ANTIGONUS: Hear you not? First Citizen: Come, sir; go, be it must none but you: I am ta'en and leave of you; and, then, go with me. First Servingman: Why, we will, sir. Second Servingman: I would not, no more: the provost. Third Servingman: What's the matter? Third Servingman: What's the matter? Second Servingman: No, my master, I can tell; I know how it you, thereof he m it. Clown: He's here; we should be a witness to the purpose. Third Citizen: He hath done what you are done. Third Servingman: A Romeo, sir, for them he would have found baw Rome of the world. CORIOLANUS: But I love the child, Not that thou hast other of thy life. Third Servingman: If thou she were thrice a man, And had these griefing force royal royal queen; And I am son, and I love thee myself. KING RICHARD III: But I will be poor; which he is he made? BUCKINGHAM: My gracious lord, let's see your grace be company. KING RICHARD III: Why, what a fellow should be? BUCKINGHAM: My lord, 'tis a subject, proud Which else, which, had some him, it will with heaven Till wind manage our doth made over his land. Why, proud I have ta'en the service of the people, That would not have more better that, Which will be satisfied; and, how his ignorant shouldER' He seems not? and I have had rather thy, To do my country, to make the'd right, With all the envy of this loyal, Our holy and be England and Duke of York, Not in his second soul I and have been The which never bid us, and call King Richard's lord. HENRY BOLINGBROKE: Welcome, uncle; we shall not be unto him; But so, my good son, I I do further. KING RICHARD III: Why, then, I confess thee, lord, If thou hadst never yet thou yet hadst never, But yet thou wilt wert up in thy horse. CLIFFORD: Clarence and Gloucester, I will not bethink, But that's not yet did nothing: but the gods The times of revenge! RICHARD: Nay, bring me what ancient, and this hand.' KING HENRY VI: But, to thee speak commonth very day! CLIFFORD: That thou wert so disgraced me to me. KING HENRY VI: Woth he the oath that I should tread upon our way? CLIFFORD: What may your grace in poor Henry's life and him? WARWICK: No, like a bawd, you not so; and, and for the poor one did, he is even to give his l. WARWICK: Uncle, so: I do intend to him in all, I hope. KING HENRY VI: How far I mean? if this be so bold with! The which, his queen, the queen, his love; And so, gentle king, may they do you in at night And fly him in the slander of his king, To be revenged on him that, so should you have; For, by this way be you found you to, She's a woman. ANGELO: He will not see you, sir. ESCALUS: I am aTo your request in hand, you shall in arms You must give, and yet go by, a course it is thence. ESCALUS: LARTI not mock mine own good. ANGELO: Do you hear. The duke hath forgot you for what you are? ESCALUS: BUS: for whose offence? CORIOLANUS: What to you? CORIOLANUS: O, worthy madam, And have I too most, you shall not. AUFIDIUS: Worthy sir, farewell.","title":"KING R"},{"location":"gpt/#vol","text":"Had not himself, nor known she! Provost: An me, both that; he did, sir, to steal-on Her mother's Romeo! O old faithful friar! JULIET: Why do I more than that I should never speak of. ROMEO: Nay, rather give me leave me that I will; For I shall find some better happy days Than lay, a man. FRIAR LAURENCE: But slain, her hath the next day a little. ROMEO: Nay, good woman; my turn in thy sake art full nights here. Who is dead and young prince. Ah, how, dost thou find me to thy breast, Thine hast he, and more, a noble fawd. Sir, thou canst not speak, I wouldst thou wert For sleep the mark of thy deadly years? Ah, keep'st, asLord men, thou hast got By so thy voice: there's no power, it is none honest. MERCUTIO: Tybalt, Romeo, whom I have the best I think how I have done. ROMEO: Thy life art thou, that thou art too fair! BENVOLIO: Mehe, and go not in 't. MERCUTIO: Nay, I'll bring some noise for a for that. CLAUDIO: Your face? aarer is a traitor's head to pluck him to the king the house. But which you can do I have to the hand; for it will be, as I am and a block withFor a man that Claudio hath married, If be a uncle. DUKE VINCENTIO: He did know more. You are most little more, he hath done it well, our the city and do them to make me theD:' Aufidius come to me and brother; and she isome with those that hath been, 'twere pity us all. Then, soft, wot each that am aly; For now I did well know. My mother! LEONTES: What's well man? CAMILLO: KING RICHARD III: Be not so? HENRY BOLINGBROKE: So that of God he would have the king his! BUCKINGHAM: I'll make more than my gracious case: I will bear the deed. KING RICHARD III: tis you by king, I would am I bethink, To make an amissant in love That thou wouldst protest to keep post toOR'n; And thou he is not nowSo. LEONTES: Unman, it is; Nay, good my good lord, be satisfied still. LEONTES: A most business: Let me have been since I was to be mercy; Whiles the lark-f orth-she, He cannot, Marcius worthyKE: 'tis well stock no poison. The manner of the envy he spake to Marcius, That with the power is worth dead, the doth be brief, In worthy Romeo hastous tune up. KING RICHARD II: Whom was it so, my child is set on death. HENRY BOLINGBROKE: ClOR, my lord; With late that cons news, for, to that I would Trueass the great number pardon'd, from your head already're his mind i' the there, I will. DUKE VINCENTIO: You do but see your highness of your arms Would know the king's mouth with peace. MARIANA: I mean, a son, it is not a word through a tear for Claudio, and be banished; But when you bid this manner bless us, In whom we have had been wont to do, 'twas upon the business, you I shall call'd his friends to him, and then. DUKE OF Lord: He's tearsark; for long my gentleness' I let me hear; good cousin, adieu! Belike his hands I in his remedy; And I have died to face, my love to me, And I will make him leave of can say, I had rather keep their words with him to keep him Whither, to thy fortune but put off; And see that this terror may be so, So if that cannot: so that's the friend, That will be deservedly of your honour's life, Let thy rage here cut in his grave; One side do not. MENENIUS: Hath the people's great state by power, That they shall have set up a part; But, as minetis the nothing of a man, That is the deputy,tis well for fault, And breathed his honour with the fear of death, That all for that as you are, and we may live. BALTHASAR: A pretty fardman! RICHMOND: Bid him be here? and I'll make me pardon up. RIVERS: What, hast he not? BUCKINGHAM: My lord, my gracious lord. KING RICHARD III: As I remember; against thee, thou wert not so much done, That thou canst swear and thee, for thou hast no cause. RATCLIFF: I will not be not, my lord. KING RICHARD III: Ay, O my children, my eyes! BUCKINGHAM: No, mighty lord, I hope, shall understand it. KING RICHARD III: I know no? but himself he is done; The side must be seen and make me wrong. QUEEN ELIZABETH: That thou mean I sent to see thy life To have no more fit than it is in this world To look upon thy brother's make me hear, The Duke of Norfolk; if thou darest with Thy Lord Northumberland, rouse'd and thy crown, For thou shalt no life but by thy kingdom. KING EDWARD IV: But thou, in my turn, my grave jest thou, You, the newats, and have the ears, Where fruit men have done-bended, then my daughter Of happyness! O my woman! If thou didst, thou tis'st fair a piteous, which he was, And know not 't prove a thousand right while you That you are to have something: so, my kingdom, 'Tis just that I iter 506998: loss 3.5516, time 71.91ms iter 506999: loss 3.6803, time 72.24ms iter 507000: loss 3.3026, time 72.24ms iter 507001: loss 3.4185, time 71.62ms iter 507002: loss 3.6271, time 73.12ms iter 507003: loss 3.5836, time 71.73ms iter 507004: loss 3.1981, time 75.92ms iter 507005: loss 3.6141, time 71.98ms iter 507006: loss 3.2294, time 73.99ms number of parameters: 3.42M No meta.pkl found, assuming GPT-2 encodings... The state of heaven with him, and that his body is humray'd this. WARWICK: And I. DUCHESS OF YORK: My dear lord, DUCHESS OF YORK: HENRY BOLINGBROKE: As far go we as any we toOL? HORTENSIO: I think she is well: then will you have so: I would we were not been then in war's good. DUKE VINCENTIO: We shall not be still be gone and been to see. GREMIO: And your good lord, I would all and like you; and in your hope. ISABELLA: I can know the truth, I am your father's house. DUKE VINCENTIO: It is no gone, sir? LUCIO: I think I would, sir, I know say, I am a friendlike by him: it is the very much of your blood; if Would I be well show'd of you in the bosom And our new-a. Provost: MAMILLIUS: My lord, I'll warrant thee a holy heart: which's with mine, I am too good to him. LEONTES: I am none; Sir, for your son I have here in this business To the justice of your father's worth. PAULINA: It is my gr in that I were, To the most of your own: your therefore have Be consul: Come, come, go, go, be much here too much: I am no more. LEONTES: They are such a and the and yet. HERMIONE: The better, sir; I am an't that you have notity. LEONTES: LEONTES: Well, well, what! LARTIUS: I am so. PAULINA: I pray any I, sir? PAULINA: Alas, poor woman! What ha HENRYHAM: Too true Clarence was a holy bed.","title":"VOL"},{"location":"gpt/#camillo","text":"I shall have had a brother of your son, To yield your brother's brother. LordOUCESTER: CLARENCE: KING EDWARD IV: KING EDWARD IV: Why, then, think you what: go-morrow, son, That you must not do that I am just, For when you can, my lord, my brother, Henry With your brother, if he bring end to your last. And, high-ed, if you see her you do, He would not have been thus an whose speak With that most un Butice: I would, he were born, a name: The house of him I'll speak to see your wife's. DUKE VINCENTIO: 'Tis not her in, sir, his friends, which Of wrongstAName to the earth, with a little way. ISABELLA: I am so mither'd by the point of it That I have had. DUKE VINCENTIO: O, let me think; and let me see a word. Provost: I wouldISABELLA: Wilt thou not, then, not in a very kind for a very for me that I did use me from him to speak. And I beseech your majesty and your bosom And my great opinion, have a wife and father, Unto my brother father and my well. FRIAR LAURENCE: And there is my hNor talk'd be no sworn to death But that I should think 'twack the part of you In the duke's daughter. Nurse: A love a man that his blood is more a life Than this may standow make her false friends. Nay, more farewell; I not for't; Our unf shors'll ask you: come by the man-likeent of this lady is the least, But to the everatter of me dearest day, Or me but one earth, like a thing that Not so much sorrow's, as many more is true Than this in his own! as our bthe light As I do! What's in all my shame already may in openings! You, my sweet HASTINGS: Be early what would you say to our lord? YORK: It is not so. GLOUCESTER: It is. LADY ANNE: sweet my old grace, that thou follow'st! GLOUCESTER: And, for God's sake, till you are three years. LADY ANNE: What, prince? GLOUCESTER: No noble thing I should be a happy king? GLOUCESTER: Away with him! DUCHESS OF YORK: My lord, I must not need: Yere to my children's love's loss of thee, But thou, even to the king, thou thy doubt'st tongue. KING RICHARD III: What is't o'So? DUCHESS OF YORK: I, by my life, I have done, but so in once For in the virtue of our blood: without that kind Was never so far gGLOUCESTR well, And I will follow him. DUKE VINCENTIO: M King Richard, give me leave to my thou thou go Be at home of these good fairWARD. But of what, thou dost to do Edward! B vouch'd: My lords, more than you shall please our royal name. LORD ROMASET: So, to save the king; yet he's just, And come, to answer me in him my country, And you shall have you where we did say the prince? DUCHESS OF YORK: My Lord, may you do this dangerous lady, My father's heart, which your hand did thee yet be The first one of this noble Edward's king; And yet that is no more: come to thee, For I am we of my your love. KATHARINA: I have a very. KING RICHARD III: sadful lord, and his part-f ready I come To make the king a king from him there is That he's made good. Come, let you go; But there be 'twere noIC at you all; But you shall have France, for honour; if you were now have My body hath done to banish yourself, To prove a good will-dew heart, Wherein my noble prince was comfort To make a man's name and a great bed: 'Tis well aKINGBe, he's in a child I'll be by my husband; let me she: I'll have it if I use my sake. Why, how, let is in that sword this traitor! HENRY BOLINGBROKE: Richard, help! but more than his that I'll make my tongue to love thee such life As thou art to Romeo. KING EDWARD IV: This? but doth he not Edward's death's death! HEN MARGARET: It is more than no more yet than by it. BUCKINGHAM: What, is it so? GLOUCESTER: I shall be there was in my heart With gracious lord and my good lordle brought He is well: but I hear you will Give me to my duty, to the world. First Murderer: Because thou canst love, that ever, thy love, thy child, Thy man shall take a grave on thy wretcheding. Fare not thy sword, content Camillo, give Of your she on a my master and son; And here, I beseech your highness,-- FRIAR LAURENCE: eyeTYou may my kindred again! ROMEO: A good things still, this is this day's run Doth never speak,--ces nay, For I have seen thee very like a fard blood, This were thy poor, which bestELLIO: Let the more be put, sir, play in, I'll tell you is Edward in such deep men! KING EDWARD IV: By heaven, I hear no more: my brother's heart. HEN ELIZABETH: I am come to have a more worth in this. KING RICHARD III: Allak thou, God! it is not my son. QUEEN ELIZABETH: There is no more I swear than they shall grief is so. WARWICK: And thou and that Romeo's and thy son's face, That thou shalt thousandame in thy rest, Whose father's heart, of grace and heart's blood is come, And with the tongue that did not be there. BENVOLIO: T Gm would cause, I wouldsthip wish it true. ROMEO: Is thy life- vouch's own word. MERCUTIO: My lord, I'll take thy choice to go Hath yet with an heart-day's death? BENVOLIO: I dare not for the best upon thyPle I talk of thy Ver lives. Come, let's not A noble man: For she hath yet a man to fall? ROMEO: Thou art not so well:OM pity her: But he's an but one. FRIAR LAURENCE: WithSecond, sir, a thousand duke's of idle hand; But I am sworn to it I should not stay, To be a life as thy honour's love: But let me rest? JULIET: This will be, to put forth; there is no man. DUKE OF YORK: Well, me for mine; let me by till thy husband I have no brother, till I am not a king's son: For I have done, I here hear you speak: By heaven, I am a beg by my life, So far as most in a man that's the mour of! O, that I would be less! the great which o' the honour, And that I had; but then, I am a-beLEath slain, Were kill my takest! If never, run Aufidius, 'Tis thought it would you are at enmity That you did but so much is, and And I'll lay down out whose't be fullle In such most I have kill'd his have been such his, As so in the world is dead. Messenger: The man that he is our lady. KING RICHARD III: He kill, my lord; who, we could not say 'ld have so much. Come, come away. Of God's name and our fair? 'Tis oath? TYRSON: He's lord, my daughter; but I doubt not more, Farewell; and 'twas it be before I did Or not a one that she is not to be thy tNor in a thought of the world was not enough to be the king. Provost: DUKE VINCENTIO: How? DUKE VINCENTIO: O, let me go my lord, I would youheeio her, prince, of you, And make me be seen to the people. DUKE VINCENTIO: I do believe it, to grant and love me My state English yourself, being full of your love! O prince! one of you a man that I must not all be the king, which is mine and him. LUCIO: But, madam, an'tess'd by the do break your honour. DUKE OF YORK: I, for an Murd-- KING RICHARD III: What! what goes this? at' lie I'force? DUCHESS OF YORK: I would I had a poor day for- thou art made it. DUCHESS OF YORK: I should I will love thee any Isabel, If I have seen the king. Provost: DUKE VINCENTIO: Good Romeo, Romeo: 'twas done, As that in any man have done to you wretch with him: So, well I desire it, to my heart Is this the day of Edward's wife: Thou most Richard, I fear me, Thou wilt make thee as thou wilt be a thousand As I am, as most gentle for ourselves: To this my husband's fair day, my soul, I therefore let her stay with heavy kiss: And, as I could, but I would, my good lords, Ere you be king in oath, and my death. LUCIO: Why, how thou hast done this man with thy love! Farewell! thou dead too, what 'twas no time Thy other blood should come; and with that WhosePOL know to fain: 'Tis so false, to beg mercy, their pluck'dine side For their goodILL'er's hands. POLIXENES: What, be there enough' the king with the king: Clarence other m O Warwick! if it be done, And that my good cousin, I wot that I know. LEONTES: How can you think! We must be this be you; this power are My true son, my brother. AUTOLYCUS: Here's he that did keep her to be: he's a par and a the duke; he is very well said, he doth: He had not so, I do fear. AUTOLYCUS: I may say, sir, my queen, dear love. Clown: My lord, you will, by my life, a word; the he duke's daughter's wife! MAMILLIUS: I have not been i' the lie. DORCAS: I know thy canre so. POLIXENES: O my lord! MAMILLIUS: No, my good Paulina,-- DORCaius out,-- DORCLEY: So much to it, sir, that it may be received To the sons; your pleasure where, the king's heir, Have he been nothing to be depose, But the poor law so far the rest is dead. KING RICHARD III: Ay, if I have Clarence's bosom forth. QUEEN ELIZABETH: WhyWARD, Well, pray, go with us? he is very comfort. RIVERS: No, as the devil'st I am no more more; But I shall have heaven with my life to go. QUEEN ELIZABETH: How long the justice of the earth is not mine? GRUMIO: prison does: Come, sir, it is no more. And the rHAM: My flum, will tell your Lordhip; here is mine ToWARD; and yet I then lay at arms.","title":"CAMILLO"},{"location":"gpt/#york","text":"God save your lordship to his own Lord Hastings! I but on such right and said 'tis well. KING HENRY VI: O, then, I'll away thy life to bear. QUEEN ELIZABETH: Come, come, come, let no longer, call to our daughter: For now the news is come. KING RICHARD III: O, but the father that is the king, The love for maid truth: 'tis in the needful Richard' death? QUEEN ELIZABETH: Nay me, gone, go with me, in good I'ight in thyAnd shall the sound wrong. KING RICHARD III: So much theseford still he is: His name is troth thee well my death: What, that is not so much, any of thee and let me think she hath no further As, nor any soldier, nor pardon, But that was she hath done, she should be. STANLEY: It is no more of your bed hath best not true. KING RICHARD III: Hath there been, and die out, so much to me The word of a love: what art thou? Second Lord: Thy father lives, that therefore can not be I take theUTA way out of our offer, And, as I say, for we shall not stay. LADY CAPULET: O my hand, a fault will be more never heard Or else moreare. CAPULET: Under this boy, she is of the Duke of York's son He live from mine honour and his blood'st To take us good; and, with much subject Like to the cause to be thus. LEONTES: You have content'd your most mostday To speak your ownness and my soul. LEONTES: Why, the Lord Angelo, So York, as it be father, for his life, To call them hope 'twere yourself and dangerous. CAMILLO: Why, what a thing? LEONTES: Was everBut in such a time of her, Or in a good will of this unisest? KING EDWARD IV: What, lords? WARWICK: And Warwick Clarence at the eyes of justice! KING EDWARD IV: Why,ere no more. CLARENCE: GLOUCESTER: CLARENCE: HASTINGS: KING RICHARD III: Death, brother, are you through my soul Sray to me, and leave me to my uncle? KING HENRY VI: This is thy brother's son in want these hence, Lest as I say, his Earl of Warwick, And tell himself to the Lord king to him, And he had been the will This comes. QUEEN MARGARET: Why thy words have there the blood for this land In that thy heart that should have. You, God's right I should have needful nothing set upon, Theself I am: and yet thy doth-- QEN ELIZABETH: O nothing, that my lord Richard will have. KING RICHARDISET: Yet, as I come! KING HENRY VI: I know the name do march. QUEEN ELIZABETH: O, I wilt thou not know that sorrow hath To hold the world with thee, and make me think How I am of thine. KING RICHARD III: In whose from all the shame of save, ThatCORIOLANUS: Why, how's there,-- VOLUMNIA: Let's hear your heart? MENENIUS: much for my very word, I'--Come, let's go: I am please her I shall do it, sir. MENENIUS: It is a cause, Pray you, get your words. BRUTUS: What, our good lord? SICINIUS: He shall, you shall be; he are three gods Is full voice. First Senator: MENENIUS: Ay, but I will go. First Senator: I know your Rome. BRUTUS: Is the news prove true, my lord, and thySo, as I And see that this beable and mostadman: Hath not a high Mercutio? That's more to say 'twere well;' which I pray I am too more than some, but so good For one to be done. LEONTES: We know my friend, I am theWhat I of your body, if But who youath done the noble duke, But what is done. LEONTES: Good Marcius, you have lost thy devil, to lie all Good night. POLIUS: The people did. SICINIUS: Are you all Aufidius? First Senator: You would he were dead, but not so much I: But now I'll call for it. MENENIUS: Well, well, no more. CORIOLANUS: As I had forgot the world, you have not a kind of love Must be a child to be satisfied, For you being no man to her death. BRUTUS: Away! the arm Of your fair wife'st, For she was my enemy. What doth the man! DUKE VINCENTIO: But makes him, good father, Thy poor prisoner is your time, I'll have a brother of a son too; The side are all betwixt me and his brother. And I have need not; and, as I am not The goodly gentleman is too, do not But else 'INCENTIO: O,en O, good me, indeed and I'll thank thee not no He hath deservedUKEill live to do this; And, if this beNORine no man's death is cold. AUTOLYCUS: Away! he is not made good, sir. DUKE VINCENTIO: My heart, my good lord, did't not't. ANTIGONUS: It is of this; We know of you are now. Servant: My father is at hand, Who came Camillo: 'Tis goodly: I'll be the bosom on the world: When the common misfoler of this battle, To","title":"YORK:"},{"location":"gpt/#iter-599988-loss-28334-time-7135ms","text":"iter 599989: loss 3.3947, time 71.09ms iter 599990: loss 3.6317, time 72.32ms iter 599991: loss 3.5352, time 74.15ms iter 599992: loss 3.2102, time 72.34ms iter 599993: loss 3.5483, time 71.71ms iter 599994: loss 3.5832, time 71.76ms iter 599995: loss 3.4675, time 71.01ms iter 599996: loss 3.3202, time 71.27ms iter 599997: loss 3.7681, time 71.95ms iter 599998: loss 3.6641, time 74.20ms iter 599999: loss 3.8146, time 71.89ms step 600000: train loss 3.6565, val loss 5.9145 saving checkpoint to out iter 600000: loss 3.3908, time 193.50ms number of parameters: 3.42M No meta.pkl found, assuming GPT-2 encodings... The state of men with him that, by his brother's, The very good men, and hisWhen we revenge to Myself, like being now in the airing. GLOUCESTER: This gentleman is set out of other. LADY, lord, give way, and go we to our cannot, And I will leave them to one the YORK: Why, then we may come to you to the: I would thou wouldst hear; war's still he thou done: O, is 'twThy graciousYou were at the heart. First ay: I did not stay? HERMIONE: You will not live But sir, nor little's by the he'll take it down To can. You are not fit to me? LEONTES: What, youPEY? MELMy words may bear't. The one would have so. ESCALUS: I cannot speak in heart to come by with her the thee: never only of your love, O, He has with him most peace; he'setheth once, And therefore do you any thing he would have you; That, in't your eye, well- ELman is so long a Jof which's made youity. If you, you may hear that you are, wtis a shepherd only may he say Bohemia. DUKE VINCENTIO: To your love. LUCIO: ANGELO: I am that king. DUKE VINCENTIO: I would I had been on to the A there: Yet must you be, sir, to all I know so. DUKE VINCENTIO: The duke shall know you say speed again to yourself you. Provost: I will be an't. DUKE VINCENTIO: It is a word of me. ANGELO: TISTRESS OVERDONE: Ah, howio! not all as he being As is a father of a e'er these adTan your brother is born. MAMILLIUS: I am a word to-morrow; youpt, But you might, I shall have had some to him thus. LADY CAPULET: If you were mad? 'Tis full of what you should have. CAPULET: What, say'st that I mean to me to thy name? JULIET: I am the more. Nurse: Is a word. JULIET: What is the news thou bring? ROMEO: How shallest what's a d see of them! ROMEO: O, give mewence, that comes in the time Of this. I am not born a happy day: The which is an ear thou now to make me speak, Nor how in that word, thy majesty worse, To Romeo, not to theage, thy dear queen, I am a comfort that is my wife. You that's a brother, to the duke's e'er, To have his mistress king, to-ake: This is the love I might have doneish the country, That I may live in theWSo name lords that means That ever had been to doway on! What think you, if you were you all, Will not my full very reason for a great bed, Uniansch high the heavens? answer this Of this fair prince's name? CAMILLO: I will, my lord. LEONTES: Pray your father! That's true-Sir, and you are son. POLIXENES: You are rough sir, give me Lord: let him go your seek. LEONTES: My fair lord, Were I but so much, or do I see, my more proud, On this most gracious lord. ANGELO: Thy great youth is the forth knows. ISABELLA: But 't, an confp Henry!' The which you seem not you both, which would well, That I with dearLE RICHENTTER: As now I have nothing: With quoth so, my lord, and all at night We cannot speak, let it to remember me. CLIFFENCE: Is some thought of this, we may be a cause.","title":"iter 599988: loss 2.8334, time 71.35ms"},{"location":"gpt/#king-edward-iv","text":"LADY CAPULET: And to me have me bawd, youngly, I would say, If you say so, it shall know the world thou thyself. LADY ANNE: And, I am that day to last with a thine. JULIET: O God's sake, is come! I'll make a years upon thy husband; And, all am grief of call thee all this good. QUEEN ELIZABETH: It is my father's; it is more A dead, a proud; for if she be king, I may not live another woman, I am not a father that will take it; And now, for my father, by my whose fineoe Provine and my wife's son brave father me: But I hope, my lord, for I was seen, With mine own love to-day; let me lest: So I have done, yet comes one. DUKE OF YORK: Thy kind-morrow spake for the grace-night! DUKE OF YORK: ay Richard, that dothst thou do to love thee? DUKE OF AUMERLE: Then, I am mine own again, or I, The truth of day; and with a kind house Hath made her dead, who have manyre many more That I'll hear him rather and say, for my death, I give a purpose. LUCIO: O living, my lord, it is gone to leave it, And you shall have of this good or'dN At my poor brother's kiss: and I have seen, The one that's made me welcome. DUKE VINCENTIO: This oath have as high as the wisest, then other mure of a chide on: the post of those: You have not there here are comeable. LUCIO: But in this while I should have them; if I cannot, I do put the fool: then, like I come to love thee With a set upon my master, which is his pence of the king; even so in his hands, As I havech you at his best, By that it may France, for honour! if you were now have all; I will be banish'd, do not to prove The corn of honour, and I will not The noble touke: but I will not have any name That I, and think it were, if I could stand, Most might be in this fair way thy hands And me shall the face my husband'sJShould be, To young princely to thee here, be long-s, Take it again in noCOR'er a happy depb not. ROMEO: O Romeo, nurse, how can your dU is a happy In mine honour or anBHAMELLA: He shall have a poor way inch by his honour Gless worthmen; I warrant my counsel was more. DUKE OF YORK: Let me I speak of such, my lord, That very good to be this, before the time Wept with Iixt these words. DUCHESS OF YORK: Why, so I am his wife, to do't, Yet hear my lord; 'tis so; and I, I do him! DUKE OF YORK: Int note was that ever wasR pleasure true: He which's anMENain'd now: heuck me not his bawd than I have content the old gold-bed, he's dead, That may not beoe to be thy face. DUKE OF AUMERLE: Whom is the way, to youQ is it so; For I will not: so she would thou hadst no time Than in me but not: but that--tis not so, For I should, I think, I please you, my sovereign, And by our country's breath shall hear thee, To lose a matter with a word; if we At the most little: to such nor thought it was, Unless I think had been yours, for I crown'd; And, with a word, poor soul, that thy love's will do it. BUCKINGHAM: I know the gods. GLOUCESTER: Pray, be these my bosom: for the time I'll pluck the pack of your house face.","title":"KING EDWARD IV"},{"location":"gpt/#king-henry-vi","text":"WARWICK: And Warwick's death will go and did to us. And yet not be fear'd in all. Go, Cates, and not thy brother and brother, And come, I'll swear the duke a good To be as free as little as, so myTis-- That side that would kill'd my husband, so Whose honourFor it is lost grave in-- Whither dost not be long as long as Henry's love My father, Edward, to my tenderbr own arms: Till your three days give good thine and your eyes. KING RICHARD II: O I wast thou not dead, say I know thy mind, Yet thou that dost in beseech at us, And not to the people that brought you to me And think you of our good rather shall be Be it to me but for my best good. First Senator: O Clifford, O, it is the more! MENENIUS: I would he had a people, But, were you not't; and you, my lord, You am of what you shall be so? BRUTUS: When you are should have bound 'em. MENENIUS: So, most it do. BRUTUS: Where is the love of this fellow, that must not? AUFIDIUS: We do not say: We were to do't. CORIOLANUS: You have been a cause to need. VOLUMNIA: O, indeed, I'll to desire o'er a counsel, And I will play the us. MENENIUS: My noble wife is me as good as well in true, To have an those water's better and isar'd. He pluck'd with him, and you'll have all the news-- And it is as good as Montague hath. Second Citizen: That, my master, which is his friar o'er-s, In that she had done the noble duke's from The noble head to the-night bed-like. First Citizen: Sond to the king: But I hear, madam, let him be heard it Of God's name, that is ill and great blood On thy be these; yet every w thought I with it, And none but that I doubt; and, in your love Is it right: but I, my poor dear cousin, I hope that I had not to this brother, But I will stay in the thing I say. DUKE OF AUMERLE: Why, lords, what aunes may? DUKE OF YORK: A gage, prince wouldst thou tell thee they! Thy brother's death! DUCHESS OF YORK: Unshe's aHath To Romeo's name the traitor; And you, God and I will see thee a and that Of living men thus they had been in this love To the duke. QUEEN: So, they are not to be the name of all. KING HENRY VI: But, madam, be content for thy thank; If thou dost sleep at thee? Now long rest is dead; thou wert not thy f after? QUEEN MARGARY: As thou as now I love'd myself of thee? QUEEN: But I shall fight with him. KING RICHARD II: O God's pity where long the o'er the world-and'd ! within I have done for the hour of thee Whereof thyself be too quick for my death, That enemy, with thy warlike blood, And made the king in arms at gone. DUCHESS OF YORK: So shall I do, to so late, Why, proud-hINCENT are lived told? DUCHESS OF YORK: No? KING EDWARD IV: I know you that is a father, to my? YORK: By fair, I thank thee, love thy face Which thou hast welcome of the fool, thou thyPR; And, now my poor queen willBYADYOP, For in my brother I king have play'd heaven: For Edward's face, being gone to Bolingbroke. TheAU's aH no other tongue to thee; For I have sent not to be of.","title":"KING HENRY VI:"},{"location":"gpt/#duke-v","text":"WhosePOLIXENBERTER: Not that that shall tell thee to my grief, But I will not be gone to thee, and thou a present tongue. 3 KING HENRY VI CLARerer: Take it with the day. BUCKINGHAM: Therefore, my lord, I let not beauteousors. FRIAR LAURENCE: Why, then, I pray thee, call thee bring me to me For this my gentleman's letters and her: 'GoodISTR in't. ROMEO: You must beL blood; And thou wert dead with a for him thou this! Nurse: Go, come, come the nurse; I mustETH: I pray thee I know thou canst wish it not say. JULIET: Though we have less, for ourHave set than me, So full of honour, I am gone to Warwick. ? of ourmen born? Second feel so: this is well as true as you, That you are put up and see as fast as fit As far all-f- HINCDand children to my poor blood To this loving? LUCIO: Doth she, when her father's Claudio hath done. What's he? He is in the name of men, With tears of heaven, and their true-ouon put'st Ourself'st man; and, and so is they all; That thou, like an unf swift use of blood. O, I have thy king, and thy king, That thou art a rest, thou, then now in thy place, Take thy hand in thee to do thee most am dear. JULIET: O do this love, and be an O! O's thy title, Is more more than yet thouThis: now, and that thou art To be it? 'tis not a tas beg man in seen an hour, having whom, he will not tell him. The means of you, my lord, is your lady's son. First Citizen: Away, my lord: we know they do us all. First Citizen: Woe to you? then you have said you were?","title":"DUKE V"},{"location":"gpt/#second","text":"Gentlemen, may she be put for it. I but think it was a bawd, I had to do so; For, in good time, a very well-baww gracious. Nurse: Why, how now, I art gone, Saw-b, ere, I hearon! JULIET: OGLOUCESTER: Here, that would he were so sour to he. LADY ANNE: Why, Warwick, how now in day shall I be! GLOUCESTER: gone, my liege, in what! If it be I had to die their high. KING EDWARD IV: Then am you patient; and therefore is Of all my husband Paris, to-day. GLOUCESTER: O, I would I had rather, but for thy son? KING EDWARD IV: Your death, my good, what! that I am there. GLOUCESTER: No, to thy name; but very it is not, For I have, to go with me; why, he's dead? heir: We shall not, my love, my lord; But, Warwick, do not justice her out; For therefore, so it is his hand that is: And yet our country, we are, nor I. QUEEN ELIZABETH: But what means, good father, be that state? KING RICHARD III: Tell 'p thee, an brow of all. QUEEN ELIZABETH: Good prince, so! KINGain: What do you thought, I hear it, I know you well: Your face I am not in your good counsel: Then be it then? KING RICHARD III: Ay, but he, but he's no more than a country. BUCKINGHAM: Is it none, good father; you hear no more. KING RICHARD III: We have done a better, our person. QUEEN MARGARET: Call so it, lords, have made him our hands: Berely as well as I have; for this, As for ESCALUS: I think much; but they find't their way out, but to die The people, and he hath yet show'd! ISABELLA: What you, I'll givehe outOUCCIUS: I am in need. ANGELO: I was a kind of all. CORIOLANUS: My gracious brother? LUCIO: ouson? ESCALUS: UC.' is there a thing? VOLUMNIA: I would it, sir. COMINIUS: Let me but stay. CORIOLANUS: The gods too word, my good nurse, I come from Rome and of the had the gods alt have been in the presence there that made; The people lives upon our TheABes; So therefore you, I cannot do you, you Were, my good with me, your father, his lady: One of you, he has done, by the honour To Coriolanus. CORIOLANUS: Haply, I mean to us and on. VOLUMNIA: ine, Lest, take up the duke out the traitor's son He was a like a father; who made My best son: I will wish you his love. Wilt thou? What's this? what? will this, so I here, Were in me by my master, I am like. JULIET: Richard had his grace, Nor, since the life that it hath most his, For much more shall be, and in that word. Nurse: JULIET: O nurse, I fear it were night. LADY CAPULET: Pray, you have all this day a may have youak. What, is not so long, you would have been long A noble deputy, which you come toius; And you the people is, you shall find again To see you knows the place. CORIOLANUS: Away! Second Senator: Come, I pray you go to you: you will hear me: Come on, madam.","title":"Second"},{"location":"gpt/#menen","text":"DUKE VINCENTIO: What, if you do not, you should be been well That you shall have; and to the alone of my heart: Do not think that it is hard to take. LUCIO: And in that world where he cannot be; for, I do see You that never, I have heard, but not go; I'll the souls of more and more That is a good way to hand; take TheSIC he my C art, and myself to please. Good ladyENS are thou what an openon 's tooNot, nor what can part in this fair speak. NORTHUMBERLAND: Come, cousin, you standNow down toWhich as these blood As thou wert so now in a comfort's hand. KING RICHARD III: My dear love, my lord, my sovereign liege, MostFrom earth I come for Edward's death; And longum thou, the fault, but not thy lord: I look'd, to be fear'd for me to live. GLOUCESTER: I never will, I'll speak this. First Murderer: How now, like! what's the day? CATESBY: Marry, my lords, shall you seem inh nor wits, Or take you by the once. Second Murderer: O God's brother, thou art mad, and begish'd. Second Murderer: 'Tis never gentleman, but what of my counsel But to my will at morning. Second Murderer: How long as I? Second Murderer: What with a name, was this? First Murderer: Alas'Twixt me, and must be thy daughter: Doth set the world upon thy counsel with thy thee o' the face, thou hast made them aHere. Second Murderer: 'Tis more but not, dost thou havest Lewis, That he hath leave of more unto of this Richard. CLIFFORD: WhoCAP thou wilt fall so of he had been of! RICHARD: And bid me stay. RICHARD: Messenger: Give me thy oath I","title":"MENEN"},{"location":"gpt/#1000","text":"iter 997: loss 5.8675, time 724.04ms iter 998: loss 5.9554, time 722.33ms iter 999: loss 5.8129, time 730.23ms step 1000: train loss 5.9102, val loss 6.0530 iter 1000: loss 5.3077, time 1634.94ms wandb: Waiting for W&B process to finish... (success). wandb: / 0.001 MB of 0.001 MB uploaded (0.000 MB deduped) wandb: Run history: wandb: iter \u2581\u2582\u2582\u2583\u2584\u2585\u2585\u2586\u2587\u2587\u2588 wandb: lr \u2581\u2588\u2588\u2587\u2586\u2585\u2584\u2583\u2582\u2582\u2582 wandb: train/loss \u2588\u2582\u2582\u2582\u2582\u2582\u2582\u2581\u2581\u2581\u2581 wandb: val/loss \u2588\u2582\u2582\u2582\u2582\u2581\u2582\u2581\u2581\u2581\u2581 wandb: wandb: Run summary: wandb: iter 1000 wandb: lr 6e-05 wandb: train/loss 5.91025 wandb: val/loss 6.05303 I my, your the am; By areU he: WAR H That a me, I am father Or: And to her I I, I ROM, lord:ANG is IET. ITER, He Hold can for my your am, And if yet be'll to the the a life the the he a my; P: Sh, have this, my, A,, A thee- I a tell: O. A be any, Your you, I my death For I, this. You, my As I and the PET's artUS, My life than. Have see! and hand to God of stand, With, a many me? Where of have think, it to I have not, That that that pray, As the my I not thy the she lord shall a thyIO and would poor, for the: and: What and he, ' so it, did a will his king, a is Th: Butour Will, Were,,. She, COR the her as, Good When. For lords. and butET is there your me? and shall king the this. C, thy mine. O The shallUS, That: GL: That, That, Then, here the But, What: H the, D and, NOR toEN! if must, The the,US? To my the's poor. Why's will have he'll can how man Some in cannot than come, Even a in, him. Is you anyENT me; so say, be sir, the the Itis the, be'll man I them. What. So thisAB he, That that the will, That the'll:I a king, That ' Would Now, have my the a: To kingUM, I'll b the must a do thou, H.","title":"1000"},{"location":"gpt/#were-his-thy-the-the-the-ill-or-all-cannot-i","text":"To No for they not such time, name? And was have to not me, Come of must know,, I be the yourUS his such I be my. How, For me the have have: ROMING To to when to, My: as And men, An a B all or And thou is, To man, And the: Come, am sir, An, And: Lord queen for, they my, that he part This! It in my trueG do in: Is? And we most, that the BKE: Is an NES? From say; you and, to's knowING your I his But V for it: I lordUL from well, Go in me, it the be I was. but: the her as IU the he our life; if his I most in I the is, I But in hast: Thus and your noble the do take she be: ' I'll a I your d ever, be not allEN COL to aOL, and not he, you is I make, This you not? I And, beELL what there: Do her thy here, not not Not crown your in a that, if death would we you not the w made of his me and butI not, What, H it P can that The the, So: COR H This: if'll B, butET with will hand, I the a N to must a, If, Which I would your- have by the am isIO, What no come is This aIO: MEN been d: WAR in a not, '; name: To sir, I most NotUE men, I What and d me:. my? The's have a us he the I lordose, You not did, Here, do an a she, my we a he: BR thy Why she. P love,, COR this the that how, Of what out H thy: COR his me you, And as NOR you as and that But I Yet. I no crown, say: As have: and wereTER:To: KING KING. ROM be you my the yet, some one? Second their a theI, the father this it not the his. A as not: ' it thou I theICH. O a lord.I it. That theES to he So. But! That not do. I: And your Q. I What IIO, is lords, he the stay that my come: How it were. H not, D. And, that be'sEN: H P: And my that have that not't to they the thou thy were If in; and the for know N In: for my the- never do tell R d it, Not, First, of I Come I be be hath a be her these me by this: and the it goodUT As My when, MER must. H And have the you- sir: and your To be aT, which is; That one shall must you the a, That make I that No'd. This; If now, good dIO most you time, My. N you, and this is have will am, ROM of were shall lord toUS, ROM is be his him: Sh: and her their: H to beEN I the name, Prov not all the you thee, The it, that was,IC How, As such not no IINC, our I and this is not, C. By but I W will as his Sh! you, To him: and say; she. Sir at notEN. Your me, for what is, Q them: Is Sir, one a him NOR with ROM mother that be that see in, H. I D, the so,, This man to She come? D Who, but; by He, H, I I I should my sir. As, take my as or. Let: Is Where his he, No his you be D have,And you He that in man thatIO And You in blood should That: As was lady. Now with: First, theICHEO isI very: Ah, will an- I blood him a, house, that part, my: MER, so, And I but you: What: Your come, Q, To, Now, will he see: Be, His to his an shall then,-- I. I most, your the this By will'll and For a the a a would he At I'll? I: Un my head; They thou am, I. It: To not, my noble in so, and O. By art? Let, they is have thou: COR; and I The But Th ' An are letOL my. And My will, What me, A, A good Second. Then: Sir:To am art D. I the thetis, we one. I Lord Th. D, let, Now, MEN, the, To which And. TheUS so that an be give, be so, Come then him. That can the:Y with by; you give to'll A this to be the letsIO to be the And have'll, These is a thy! I And: OfEN cannot'llUE with he my do The did:","title":"Were, his thy the the the I'll, or a'll cannot, I"},{"location":"gpt/#of-thou-say-so","text":"be, best The our What: In can should of a he I To, Which J: I: He of his sir- am, If sweet the Now. ROM To, their. men? D, would king: And And one And'll himly name. COM, sir with, And: N am's that you of there thisOL heart: You if son But: WereET: if I never, Your in your not? goodIO, or: How Give not in; Come yourEN have most: H:So the I do, If my my I And the good I this or love in think: My Come's and some not I call, to mine: KING you '. Of To But life, be, B you, you, And go: Good theUS and love W forTER to'll II them:But'll an. If ' will, and's part be bear here, A: We you How, Make thisEO's his would love, a when not: N him it, are you, a thou then, How. Cl a a for your! be the part, And take the But, sir, That: Ay, well! This, Second all-: If a I the'll all withUS: Totis death; sir of: And his her thee to this'd: Un, see and a KING, The the be; thy man, some do: And thou ' the the some thy To the this As. That--IO, The I the aKE thou in give! It shall some thou have I I I you the a To I d not am the thyKE a my the this he: Would my life,,, Well is thy, Which: Which a well. There R, the a B so: and UE MEN not in I tell withI if If in the R AsUS. but I the me when you, Good And him. b poor, One MER: To all: and is I makeant upon it I we a blood I. You Is, the, myIOL would First R, a he, Your queen: no people? TY: I. Is shalltw: And 'ES youtis Which his my your my art we haveUE me S his but man, which come: That H the your the, b, His the a a my king? And Go to his Second hath I his more man up, That, What. And do'll a a As He in he have, I my willIO: And, That he I isU You, But. you? Which to a the a, To You a, have the my him, a these this and a him: R most love: that no do make if her For I know, my grace. AEN the am your your the I lord? Who, I the with But: I, a Ns to have so such: KING: J a I day is D crown'd, That But the the. Against when it J. MEN a the. If It love Give theKEant, E, go. Sh: I ' as I ownUEAN: Thus will myIO of And, Of queen: IUS of my lord! it, He would thou? His, And that mother; And N I have, for his if shall, Now, to the am, there that great must betis he D this. C the do, To an, As a them,: ROM, Third, First: For the see,","title":"Of thou say so;"},{"location":"gpt/#i-the-a-a","text":", Now I was,'s that I it you the done Th with do my, That that it: O: Your, our'll And the that ' him like but, a honouron. I dead to her some I: He, If: KING: R thyUS. And Too: For a be theELL, our, And many the but am this, Why, They that the this make, I: Is have am her my my he all,, when the will the to be, This, As may I my their have the To him: H know is say YU ETER, Let. Now. A give is of shall thou, There you, P, I poor, a the some have the your are the for do for you this to your a for or good do, or noble: But And shall he be And good them with Cl's The are a do you. That notown, Sh, I be do it,U O you I be die, Her, by I; she, To be to that sweet all, We be: D more. For the WAR: a the-'s she, that And a thy what, And It, and these. The thetw's know. Like of makeIO. A But I speak: with and: not, PR: Come we If his,EN hear: To I I, N; Of, I, First so the head: To that hisI be you the: That you. You? ' shall I be As He is KING, come KING, the do the the So, With, And shalloth such me,, you by: That to her for I like. Our, As in this love:","title":"I the a a,"},{"location":"gpt/#000","text":"iter 4990: loss 4.7843, time 730.76ms iter 4991: loss 4.9219, time 722.85ms iter 4992: loss 4.6760, time 731.14ms iter 4993: loss 4.8503, time 699.10ms iter 4994: loss 4.7151, time 721.49ms iter 4995: loss 4.7852, time 687.31ms iter 4996: loss 4.5192, time 732.55ms iter 4997: loss 5.0381, time 727.84ms iter 4998: loss 4.6365, time 694.68ms iter 4999: loss 4.8973, time 691.50ms step 5000: train loss 4.7097, val loss 5.5308 iter 5000: loss 4.3929, time 1421.90ms wandb: Waiting for W&B process to finish... (success). wandb: - 0.001 MB of 0.001 MB uploaded (0.000 MB deduped) wandb: Run history: wandb: iter \u2581\u2582\u2582\u2583\u2584\u2585\u2585\u2586\u2587\u2587\u2588 wandb: lr \u2581\u2588\u2588\u2587\u2586\u2585\u2584\u2583\u2582\u2582\u2582 wandb: train/loss \u2588\u2582\u2582\u2582\u2582\u2581\u2581\u2581\u2581\u2581\u2581 wandb: val/loss \u2588\u2582\u2582\u2582\u2581\u2582\u2581\u2581\u2581\u2581\u2581 wandb: wandb: Run summary: wandb: iter 5000 wandb: lr 6e-05 wandb: train/loss 4.70968 wandb: val/loss 5.53077 number of parameters: 81.52M No meta.pkl found, assuming GPT-2 encodings... Not that; You. And it, I to the mother on my son.', PR: And the suour of, Quke, That, not is this earth And not, Now more years will the part: The hands; Not, My son, To I do my queen. TellY: Darest by strength and. And, As Provife's side, And I not the gOM look, which: AUTUS: May not men to dU and you, So, LUEut's lie on a would I can! The suilman, The noble lord up I I do nothing in your end, DUCatter thou have my r LA, For, with to the earth to your wife, Not my or not his hands With you do a friends, His loss, Should have not, as Ibians My mostt to I lord, the world, take you not that you the cause, IOL OFCES: To you, if her my father, I can be your Tower To do my name. That the hands to me. Icius, LE: Why, LUCIO: LELCUTIO: To thy lord, to my day with me? And you more for, For never my power, Her heart; DUE: Good great noble father, this In to an father? And be do for the friends! The lord ClOL OF YORK: And he will he of my hour. And if Will some thing at the lord Bove's father have her; Darep the counsel. As will be those the one, go; come How me: Madunes! By me. And a curse of my. ' I hast my lord Not, And, my lord; He. O Do you have be With blood, R VI: Tell your foe is a king-eseous death, Sir a most mind. That: Who I have ProvUoWARD But, O And in a go, From sweet high is joy upon my, FirstICan had this? WARICHCES V Lay: Come about a people, IUCUS: But this land'd's father, And Could To my will; let my ancient lord Unless my lord, but That, And I shall be I, farewell, if I have said their head, And yourly the time All it which thou have, Which not will; an eye! In me of her not. How I would I DUCost it have the lord, to you is my d'll my common d understand What will not.The business! Glouace; and you, Why your king is your great And Mess not's in heaven on me Say It thy day.IUS: And FirstIC VU, Be that I? AndtwEN Which Best you no is to thou should I do not, Be thy life; And and If you. And I not. A: And he callWARD: To their most we do yours, Dost; Who the first: To be't? A: To my king but Once will But. Thou hast our loving dADWARD IV: The Duke; Sself is the father, and be make you Why, Blbroksarry; From a Lord of that she? As this, Which I have but I spoke, since was the lords, Well, And your course! Sir. Do this grace they were in heaven NEENIUS: Look's, BowY: With in the word. L: But I wost to not the heart To all thee, LU' the my heart, Hre the heavens. And he, we art shall be be your other the last in this quarrel. B me to my royal prince, My life, it may bear hist; SecondUS: But in your, good day, And you, that, And it; it, to give him, From true may be not AB, Look to you will My, In thee-OM do the less aself, Before the, For I not better 'er she not you here: And be a heavy un Senator: MENall him It, It be my suit KINGELBY: Yes! Is my father, To the best, a grace. YATHolding, but such me I cause your sovereign! KINGABILLES: Nor what shall do a ear of the word's tribricMour him PER be. A: Sweet sister, My self, That the grave, The lie' so thy case in put, And my wrong, MENre father. Your child, I do be our will Nor your That KINGOU RABit's with a sovereign's heart as, and been But, good lOW: From your court will you me they friends withESSyly I-US: On the mind, Iie, And if That the good thing the very poorICIN With this is him. POLam'd me Say, And thou I do be be all my word, be my noble day, a Tower to you, Or, That them, sir By my sister; HILLES: Or than was this, Or it, LUS: For none. Have all your dINGS: HULEfore. Of her, Who Th OFABall Burse: IUS: Dcius. But time, when I will be you it the lie me, So me, So my lords, Be but bear all more His better I have thee to not by thy dU me's more may, And he. Go, That the words, which I besinks would he'd in hisENTumerine! Had I will, There has, but when Stand with him, FirstUS: Why: Our father, in CO; Like you it me's lord, But,' he heaven? Be you may the life. Doth my lord, but was you is dead, AETH And see your words. Ander thy will all down, Or in my suit, And at this f Murdaced not, To; LUS: From all the very and his And I must it shall am I am you you, Which, My friend with Inay again: For they traitor; For you but an country? To had the LU, here, had That thou, do the heart, IUS: A: IUartarry. But I but MEN: FirstUS: And they not, LE OFELA Marshal: Are not to heard theICINNow, sir, RABELLARRIUman. Well.And he a eye, and.And show to I be your death have this lords me that, in the king And this a times from I would say, then, your father, FUCUS: R YORK: Now. To see with my queen, GiveBERIO: for your sister, For it. Of your land! And For, I be use their case; do have too as see his fault And I speak Who I more stay? Let the or ent not, and now all you by a gods, you I say dU your war, In me as that: The end. God no more thus; GLCES OFUant; And I, O of HENO, But, ToASIONISHorn! Which The world, Why to, Which. Who have tost it shall done.Iona me. JOHN: And it to his lord. To be dead for peace, for me; She I they as that, Which art. Which be a soldiers And a rest with an soul, and of me to all for her, MERARDULBRES: Now. CORUT'll give me, And a house, Give your gentle be l'd for Good death, KING R HEN'B the honour, To be ourself, Firsthouse. MEN and the father! ? You'll forUCKONT thee; Of his gentleman, He will more's: ThINIUS: To I stand! The head. No, the brother's this, nor so the world that in a bo', then, Most justice, So you, as the life, What, With her this breath, sir, You thousand uncle; When he the world, but And these brother the people. That I besiss? the good father; And, LUKE: Why, I, And for the king, HEN' so to And to you. KING RABETH: Th me, Cer; Let haste, now LETER: To't, And Quke! And this, I of to the eyes, In your blood we all here this sovereign: And if a my time, As my hand, Three life: Than to your power? That, FUuUCKIN Ile? LUCIO: Your head. The head, no thy people in you can your heart, HERuck thee, They be the life. Dague to me For not's kingdom, From the death, this loss, DUS: BBY: O is my as it my noble trible, and nothing,--And my quarrel would what you, being not, Q YORK: WAR YORK: Why Now, With a my side? En them, E: As I know is my other's poor They have you is, DAR LA II: That's more, for me to myself that I be a queen and be a heart: IS VI: SIO: Why and this lord, MontM the crown: May think. To this's hand, and your hands, Are not I not I not's very man, He you, I be done, QUC love upon the cause of my air In you I he do pass, FADESSINFor my father, Bret to But here, sir? WhKE: We Q be, FU, LE: DUall what and, On, LU, you I'll, There, LUS: And you, Theark any? Which no that, COR thou be in 'er And There in the child, LARDELLOU VI: If plWARDASTICK: GL YORK: The crown, as my VolKE: Lome, For I come thy life, And HERESSou I to you shall, on true have-ighth up And lay Dague, Y: GREY: Who, To have your ears's lord; that now. But than I thing! LE OFUCH'd, Your death. FIO: 'T: To in the soul, sir on the word. Like OF YORK: GLARD III: The air, As There? Peace; What a love, FICK: but is the end, And were the friend B thee, for a king, ' he: And And that is an l me to.IUS: So, SwESS OF RentleY ANICou we did art your breath, weENTona? Say It that, At your man and His uncle. And I have him, And thou look is thee, And this very good, that go; that to time? MENENES: And here's, sir, Ay: My prisoner's me. My daughter! IENCE: This, And your restrah is your great! LARADEW' the other fair land.IAD me you may we be he live, KINGELUTET: And many hands, And he thou came in thy sea. This your sister, And I set wABiful; While'd, with this cause, and done, DAB II: Or can our breast, Henry shall were my oath! Which thatness-- T, She I have live in some man off be me now not may my lord, In a people; Which ' the words to his, If mine you should my name; I have Now in a lords, And my life with him, KINGUS: And you, poor looks's you on your father have light, If 'em. GLUions when men my On your lord? But his grace. Go,, CBRESTER: an sweet LUge, O, with your world. And I well to true is the Lord. Lordugh, LU and how you; My soul, With, And What: And, To-US: To that is your life as in. My hands. And it it to be Ct is our soul. You now, As both a queen, like once and any more, Like you be a brother! SUMGABENI would he, and I see a blood We do Clarence Well me, To time, we spICIN GRE me-ICINJENALIO: And the quarrel, 'T of a my the Tower, CIUS: On the case; Let him, She that shall be we power! To be your, And And a right with that'sENTOLou you he, And that! LE: DUiol you. And, Will be be you of mine that we more for the father. And IENTIO: For to right upon the death'd.IINGS: Now, His field, I be my gods's land, FW'll. KINGOUES: And thou If not to the father? HETH: LE:-- Look must you Who know man by her. POLopit you it man's to the other were make my d my lord to the one. BYts you; His time's more is this eyes, IS RARWICK: LUish a men is to such this brother, O, That must he it out? We can be There him that more's more me, eunes, And IC I not is your the lord. Ander before it. And thou were, a peace, And Scak down with a, AndT II: Which and him the king; QAENI had me to set sorrow, He with the lord of my wrong. He, CY: And were a gentle lord, And I had justice. He, aICan's to be your prince. The better and once have be have none! And he is him, I may this: Yet come are the king, Andn all. From our life. Shful blood'd, and not Shre is what that, Jutful men all day, my better you to your own earth, I do LICHCBRHAM: Who, And my night, My life! What this name. With the side: How And our further with ourselves, If are for the pleasure: And is; The own life, The father; And I stay. We be the fault, FirstUS: F mayENTfm. Of my death it, Firstoler. IICK OFICHINC scorn me, ThINTo the heart! Of thy while man, of him To take me, You. It by those the in me, As So no world,-- To put off's man CORtis going's land, Her counsel at your father than both you, Why more; But you, For most world-ICanly Richard, Make no need is me thee; The And so! POLKE: DOUO: FirstUS: Thus, Must shall not but I, You, lords, farewell And will not. How'sbher dost, and be, LUEENO, even or so I For in my queen's good king, To fear, The hand and, to my man: And the voices, Here aem word, And will all to a dUman: Who by her with him.","title":"000"},{"location":"gpt/#jricest-a-bo","text":"R with your IIO: And we in him am the lord is. And As To not is great Have you to in it, Mess so is my high on and the time! That I do your lord, He shall made? O you of a best, theyt with her of her must with the high me, and the master at my brother, if it you not not I in the suful house to they desire, hear fulled thy grace And your FirstUS: GLOUUTIO: O. A: Of, This it with? First should for theself: To the hand, Or you, SICIN Ay, He in me again! Which go with the time. You come's! Here myself, L: A: IUS: Or that he your blood's and he are And know your king. And With't will a son, DAD your is an old's king in him to thy house; While yourt in me, Are bear, ' they. T: ROMO, QW: DOUES ANIO: And true, he a day, T: To to you than, and so; If he is me from me is my one. The kingdom in our lord; And the hand upon BolCHp Henry. MEN: For I would the earth, The crown, Your bed, I not ISARDENES: Alure, And more to; MadKE VINC say CORES: MethESS: She of a more do IIZARD: And you, And you, I turn not the presence. To gone: And I will th, I say. our voices, IUS: O, KINGABours. And you hear my face, for they more I will his king, thy bosENES: Sir should have go, LU, If I be none, my soul is your Duke, And he. And a more ' it for the son, His kind, He, being she En is it to this I HreG am so, LE: He to be, That that by, sir, Cords, LETER: MENENCOKEANost, and not we What the wife, ESANUS: May I of the Lordus'd the That to you will not did, If For he by my man and, DareUCKou And it, But, but For the life. SecondUS: And we think, they The part. CAPishment his do been me what is you that, SecondIO: Go your do not, When you B this life. With her, if he to fear. The grave! The king: That I HFAD down, all you with me. And I to't. MESSou would he, I, IADere's man Call the word Before a mother: My life, IICK: IUS: As That to the fault, Or shall in my heartous face, by that, then are it, and my headly. LABch the present, FirstUS: To, AENIUS: ThANUS: I'll out! And we the looks's blood, and be your life. IUS: Th VAENES: SUS: That's, Which in him, Who for me I show her? But me, so thy friends, as I not, To you I ThINGBRO, GLAB-US: And then. ROUO.IUS: Of peace, On the hand, But from a lord, DUed to I-ICINLARUS: 3 III: Sir, And did not can you-blood DUed, And the great head too thou should leave, 'T: To how you him, A III: Thou will not But to not, no devil, as it, but I be him; Hans, as he, and this time, You his life.","title":"Jricest a bo"},{"location":"hydra/","text":"Hydra import logging from omegaconf import OmegaConf, DictConfig import hydra from hydra.core.hydra_config import HydraConfig @hydra.main( version_base=None, config_path=\"configs\", config_name=\"config\", ) def main(cfg: DictConfig) -> None: logger.info(OmegaConf.to_yaml(cfg)) logger.info(f\"runtime.output_dir{HydraConfig.get().runtime.output_dir}\") if __name__ == \"__main__\": main() from hydra import initialize, compose with initialize(version_base=None, config_path=\"configs\"): # config is relative to a module cfg = compose(config_name=\"config\") To check current defaults python my_app.py --info defaults-tree","title":"Hydra"},{"location":"hydra/#hydra","text":"import logging from omegaconf import OmegaConf, DictConfig import hydra from hydra.core.hydra_config import HydraConfig @hydra.main( version_base=None, config_path=\"configs\", config_name=\"config\", ) def main(cfg: DictConfig) -> None: logger.info(OmegaConf.to_yaml(cfg)) logger.info(f\"runtime.output_dir{HydraConfig.get().runtime.output_dir}\") if __name__ == \"__main__\": main() from hydra import initialize, compose with initialize(version_base=None, config_path=\"configs\"): # config is relative to a module cfg = compose(config_name=\"config\") To check current defaults python my_app.py --info defaults-tree","title":"Hydra"},{"location":"jetson/","text":"Nvidia Jetson AGX Xavier Ubuntu Version $ cat /etc/lsb-release L4T Version $ cat /etc/nv_tegra_release Kernel Version $ uname -a CPU Information $ lscpu Hardware Information $ sudo lshw Disk Usage $ df -h Running Processes $ top List USB Devices $ lsusb USB Devices List the USB devices and associated drivers. $ usb-devices Force Recovery Mode Place the Jetson into Force Recovery Mode $ sudo reboot \u2013-force forced-recovery dmesg dmesg prints the kernel message buffer. The messages typically consist of messages produced by device drivers. Useful when working with new hardware, e.g. USB devices. On newer installations (Newer Ubuntu 14.04, Ubuntu 16.04), you can monitor the dmesg buffer: $ sudo dmesg \u2013follow Xorg Xorg is the Ubuntu display server. You can monitor the Xorg log in real time: $ sudo tail -f /var/log/Xorg.0.log List Partitions sudo gdisk -l /dev/mmcblk0 Serial USB devices From: https://unix.stackexchange.com/questions/144029/command-to-determine-ports-of-a-device-like-dev-ttyusb0 Below is a quick and dirty bash script which walks through devices in /sys looking for USB devices with a ID_SERIAL attribute. Typically only real USB devices will have this attribute, and so we can filter with it. If we don\u2019t, you\u2019ll see a lot of things in the list that aren\u2019t physical devices. #!/bin/bash for sysdevpath in $(find /sys/bus/usb/devices/usb*/ -name dev); do ( syspath=\u201d${sysdevpath%/dev}\u201d devname=\u201d$(udevadm info -q name -p $syspath)\u201d [[ \u201c$devname\u201d == \u201cbus/\u201d* ]] && continue eval \u201c$(udevadm info -q property \u2013export -p $syspath)\u201d [[ -z \u201c$ID_SERIAL\u201d ]] && continue echo \u201c/dev/$devname \u2013 $ID_SERIAL\u201d ) done https://docs.nvidia.com/jetson/jetpack/install-jetpack/index.html#sd-card-image https://docs.nvidia.com/sdk-manager/docker-containers/index.html https://docs.nvidia.com/jetson/archives/l4t-archived/l4t-3261/index.html#page/Tegra%20Linux%20Driver%20Package%20Development%20Guide/updating_jetson_and_host.html# https://docs.nvidia.com/jetson/archives/r35.3.1/DeveloperGuide/text/SD/FlashingSupport.html#flashing-to-an-sd-card","title":"jetson"},{"location":"jetson/#nvidia-jetson-agx-xavier","text":"","title":"Nvidia Jetson AGX Xavier"},{"location":"jetson/#ubuntu-version","text":"$ cat /etc/lsb-release","title":"Ubuntu Version"},{"location":"jetson/#l4t-version","text":"$ cat /etc/nv_tegra_release","title":"L4T Version"},{"location":"jetson/#kernel-version","text":"$ uname -a","title":"Kernel Version"},{"location":"jetson/#cpu-information","text":"$ lscpu","title":"CPU Information"},{"location":"jetson/#hardware-information","text":"$ sudo lshw","title":"Hardware Information"},{"location":"jetson/#disk-usage","text":"$ df -h","title":"Disk Usage"},{"location":"jetson/#running-processes","text":"$ top","title":"Running Processes"},{"location":"jetson/#list-usb-devices","text":"$ lsusb","title":"List USB Devices"},{"location":"jetson/#usb-devices","text":"List the USB devices and associated drivers. $ usb-devices","title":"USB Devices"},{"location":"jetson/#force-recovery-mode","text":"Place the Jetson into Force Recovery Mode $ sudo reboot \u2013-force forced-recovery","title":"Force Recovery Mode"},{"location":"jetson/#dmesg","text":"dmesg prints the kernel message buffer. The messages typically consist of messages produced by device drivers. Useful when working with new hardware, e.g. USB devices. On newer installations (Newer Ubuntu 14.04, Ubuntu 16.04), you can monitor the dmesg buffer: $ sudo dmesg \u2013follow","title":"dmesg"},{"location":"jetson/#xorg","text":"Xorg is the Ubuntu display server. You can monitor the Xorg log in real time: $ sudo tail -f /var/log/Xorg.0.log","title":"Xorg"},{"location":"jetson/#list-partitions","text":"sudo gdisk -l /dev/mmcblk0","title":"List Partitions"},{"location":"jetson/#serial-usb-devices","text":"From: https://unix.stackexchange.com/questions/144029/command-to-determine-ports-of-a-device-like-dev-ttyusb0 Below is a quick and dirty bash script which walks through devices in /sys looking for USB devices with a ID_SERIAL attribute. Typically only real USB devices will have this attribute, and so we can filter with it. If we don\u2019t, you\u2019ll see a lot of things in the list that aren\u2019t physical devices. #!/bin/bash for sysdevpath in $(find /sys/bus/usb/devices/usb*/ -name dev); do ( syspath=\u201d${sysdevpath%/dev}\u201d devname=\u201d$(udevadm info -q name -p $syspath)\u201d [[ \u201c$devname\u201d == \u201cbus/\u201d* ]] && continue eval \u201c$(udevadm info -q property \u2013export -p $syspath)\u201d [[ -z \u201c$ID_SERIAL\u201d ]] && continue echo \u201c/dev/$devname \u2013 $ID_SERIAL\u201d ) done https://docs.nvidia.com/jetson/jetpack/install-jetpack/index.html#sd-card-image https://docs.nvidia.com/sdk-manager/docker-containers/index.html https://docs.nvidia.com/jetson/archives/l4t-archived/l4t-3261/index.html#page/Tegra%20Linux%20Driver%20Package%20Development%20Guide/updating_jetson_and_host.html# https://docs.nvidia.com/jetson/archives/r35.3.1/DeveloperGuide/text/SD/FlashingSupport.html#flashing-to-an-sd-card","title":"Serial USB devices"},{"location":"k8s/","text":"This page contains a list of commonly used kubectl commands and flags. {{< note >}} These instructions are for Kubernetes v{{< skew currentVersion >}}. To check the version, use the kubectl version command. {{< /note >}} Kubectl autocomplete BASH source <(kubectl completion bash) # set up autocomplete in bash into the current shell, bash-completion package should be installed first. echo \"source <(kubectl completion bash)\" >> ~/.bashrc # add autocomplete permanently to your bash shell. You can also use a shorthand alias for kubectl that also works with completion: alias k=kubectl complete -o default -F __start_kubectl k ZSH source <(kubectl completion zsh) # set up autocomplete in zsh into the current shell echo '[[ $commands[kubectl] ]] && source <(kubectl completion zsh)' >> ~/.zshrc # add autocomplete permanently to your zsh shell FISH {{< note >}} Requires kubectl version 1.23 or above. {{< /note >}} echo 'kubectl completion fish | source' > ~/.config/fish/completions/kubectl.fish && source ~/.config/fish/completions/kubectl.fish A note on --all-namespaces Appending --all-namespaces happens frequently enough that you should be aware of the shorthand for --all-namespaces : kubectl -A Kubectl context and configuration Set which Kubernetes cluster kubectl communicates with and modifies configuration information. See Authenticating Across Clusters with kubeconfig documentation for detailed config file information. kubectl config view # Show Merged kubeconfig settings. # use multiple kubeconfig files at the same time and view merged config KUBECONFIG=~/.kube/config:~/.kube/kubconfig2 kubectl config view # Show merged kubeconfig settings and raw certificate data and exposed secrets kubectl config view --raw # get the password for the e2e user kubectl config view -o jsonpath='{.users[?(@.name == \"e2e\")].user.password}' # get the certificate for the e2e user kubectl config view --raw -o jsonpath='{.users[?(.name == \"e2e\")].user.client-certificate-data}' | base64 -d kubectl config view -o jsonpath='{.users[].name}' # display the first user kubectl config view -o jsonpath='{.users[*].name}' # get a list of users kubectl config get-contexts # display list of contexts kubectl config get-contexts -o name # get all context names kubectl config current-context # display the current-context kubectl config use-context my-cluster-name # set the default context to my-cluster-name kubectl config set-cluster my-cluster-name # set a cluster entry in the kubeconfig # configure the URL to a proxy server to use for requests made by this client in the kubeconfig kubectl config set-cluster my-cluster-name --proxy-url=my-proxy-url # add a new user to your kubeconf that supports basic auth kubectl config set-credentials kubeuser/foo.kubernetes.com --username=kubeuser --password=kubepassword # permanently save the namespace for all subsequent kubectl commands in that context. kubectl config set-context --current --namespace=ggckad-s2 # set a context utilizing a specific username and namespace. kubectl config set-context gce --user=cluster-admin --namespace=foo \\ && kubectl config use-context gce kubectl config unset users.foo # delete user foo # short alias to set/show context/namespace (only works for bash and bash-compatible shells, current context to be set before using kn to set namespace) alias kx='f() { [ \"$1\" ] && kubectl config use-context $1 || kubectl config current-context ; } ; f' alias kn='f() { [ \"$1\" ] && kubectl config set-context --current --namespace $1 || kubectl config view --minify | grep namespace | cut -d\" \" -f6 ; } ; f' Kubectl apply apply manages applications through files defining Kubernetes resources. It creates and updates resources in a cluster through running kubectl apply . This is the recommended way of managing Kubernetes applications on production. See Kubectl Book . Creating objects Kubernetes manifests can be defined in YAML or JSON. The file extension .yaml , .yml , and .json can be used. kubectl apply -f ./my-manifest.yaml # create resource(s) kubectl apply -f ./my1.yaml -f ./my2.yaml # create from multiple files kubectl apply -f ./dir # create resource(s) in all manifest files in dir kubectl apply -f https://example.com/manifest.yaml # create resource(s) from url (Note: this is an example domain and does not contain a valid manifest) kubectl create deployment nginx --image=nginx # start a single instance of nginx # create a Job which prints \"Hello World\" kubectl create job hello --image=busybox:1.28 -- echo \"Hello World\" # create a CronJob that prints \"Hello World\" every minute kubectl create cronjob hello --image=busybox:1.28 --schedule=\"*/1 * * * *\" -- echo \"Hello World\" kubectl explain pods # get the documentation for pod manifests # Create multiple YAML objects from stdin kubectl apply -f - < pod.yaml # Generate spec for running pod nginx and write it into a file called pod.yaml kubectl attach my-pod -i # Attach to Running Container kubectl port-forward my-pod 5000:6000 # Listen on port 5000 on the local machine and forward to port 6000 on my-pod kubectl exec my-pod -- ls / # Run command in existing pod (1 container case) kubectl exec --stdin --tty my-pod -- /bin/sh # Interactive shell access to a running pod (1 container case) kubectl exec my-pod -c my-container -- ls / # Run command in existing pod (multi-container case) kubectl debug my-pod -it --image=busybox:1.28 # Create an interactive debugging session witin existing pod and immediately attach to it kubectl debug node/my-node -it --image=busybox:1.28 # Create an interactive debugging session on a node and immediately attach to it kubectl top pod # Show metrics for all pods in the default namespace kubectl top pod POD_NAME --containers # Show metrics for a given pod and its containers kubectl top pod POD_NAME --sort-by=cpu # Show metrics for a given pod and sort it by 'cpu' or 'memory' Copying files and directories to and from containers kubectl cp /tmp/foo_dir my-pod:/tmp/bar_dir # Copy /tmp/foo_dir local directory to /tmp/bar_dir in a remote pod in the current namespace kubectl cp /tmp/foo my-pod:/tmp/bar -c my-container # Copy /tmp/foo local file to /tmp/bar in a remote pod in a specific container kubectl cp /tmp/foo my-namespace/my-pod:/tmp/bar # Copy /tmp/foo local file to /tmp/bar in a remote pod in namespace my-namespace kubectl cp my-namespace/my-pod:/tmp/foo /tmp/bar # Copy /tmp/foo from a remote pod to /tmp/bar locally {{< note >}} kubectl cp requires that the 'tar' binary is present in your container image. If 'tar' is not present, kubectl cp will fail. For advanced use cases, such as symlinks, wildcard expansion or file mode preservation consider using kubectl exec . {{< /note >}} tar cf - /tmp/foo | kubectl exec -i -n my-namespace my-pod -- tar xf - -C /tmp/bar # Copy /tmp/foo local file to /tmp/bar in a remote pod in namespace my-namespace kubectl exec -n my-namespace my-pod -- tar cf - /tmp/foo | tar xf - -C /tmp/bar # Copy /tmp/foo from a remote pod to /tmp/bar locally Interacting with Deployments and Services kubectl logs deploy/my-deployment # dump Pod logs for a Deployment (single-container case) kubectl logs deploy/my-deployment -c my-container # dump Pod logs for a Deployment (multi-container case) kubectl port-forward svc/my-service 5000 # listen on local port 5000 and forward to port 5000 on Service backend kubectl port-forward svc/my-service 5000:my-service-port # listen on local port 5000 and forward to Service target port with name kubectl port-forward deploy/my-deployment 5000:6000 # listen on local port 5000 and forward to port 6000 on a Pod created by kubectl exec deploy/my-deployment -- ls # run command in first Pod and first container in Deployment (single- or multi-container cases) Interacting with Nodes and cluster kubectl cordon my-node # Mark my-node as unschedulable kubectl drain my-node # Drain my-node in preparation for maintenance kubectl uncordon my-node # Mark my-node as schedulable kubectl top node # Show metrics for all nodes kubectl top node my-node # Show metrics for a given node kubectl cluster-info # Display addresses of the master and services kubectl cluster-info dump # Dump current cluster state to stdout kubectl cluster-info dump --output-directory=/path/to/cluster-state # Dump current cluster state to /path/to/cluster-state # View existing taints on which exist on current nodes. kubectl get nodes -o='custom-columns=NodeName:.metadata.name,TaintKey:.spec.taints[*].key,TaintValue:.spec.taints[*].value,TaintEffect:.spec.taints[*].effect' # If a taint with that key and effect already exists, its value is replaced as specified. kubectl taint nodes foo dedicated=special-user:NoSchedule Resource types List all supported resource types along with their shortnames, API group , whether they are namespaced , and kind : kubectl api-resources Other operations for exploring API resources: kubectl api-resources --namespaced=true # All namespaced resources kubectl api-resources --namespaced=false # All non-namespaced resources kubectl api-resources -o name # All resources with simple output (only the resource name) kubectl api-resources -o wide # All resources with expanded (aka \"wide\") output kubectl api-resources --verbs=list,get # All resources that support the \"list\" and \"get\" request verbs kubectl api-resources --api-group=extensions # All resources in the \"extensions\" API group Formatting output To output details to your terminal window in a specific format, add the -o (or --output ) flag to a supported kubectl command. Output format Description -o=custom-columns= Print a table using a comma separated list of custom columns -o=custom-columns-file= Print a table using the custom columns template in the file -o=go-template=