UBUNTU18+KVM+VAGRANT+KUBERNETES 6) Provisioning local-disk and mariadb-master install as a pod(https://www.youtube.com/watch?v=07subgHDTIM&feature=youtu.be)
HI.!
This article shows how to provision and install mariadb-master using local-disk provisioning.(k8s)
In this case, pod must be running the server(node) that has local-disk(physically).
!!!If you want to know more about local-disk provisioning visit below site!!!
https://kubernetes.io/blog/2019/04/04/kubernetes-1.14-local-persistent-volumes-ga/
Creating mariadb-master server.(storage is local-disk provision)
1)Creating storage class(sc) and persistent volumes for mariadb-master
2) Create service for mariadb-master
3) Creating configmap for maraidb-config cnf for master service.
4) Create Stateful mariadb-master
5) mysql connection test.
1)Creating storage class(sc) and persistent volumes for mariadb-master
[vagrant@kubemaster mariadb-master]$ alias | grep kb
alias kb='kubectl'
[vagrant@kubemaste
[vagrant@kubemaster ~]$ kb get pv
No resources found.
[vagrant@kubemaster ~]$ ls
[vagrant@kubemaster ~]$ mkdir mariadb-master
[vagrant@kubemaster ~]$ cd mariadb-master/
[vagrant@kubemaster mariadb-master]$ ls
[vagrant@kubemaster mariadb-master]$ vi mariadb-master-sc.yaml
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
name: mariadb-master-sc
provisioner: kubernetes.io/no-provisioner
volumeBindingMode: WaitForFirstConsumer
[vagrant@kubemaster mariadb-master]$ kb create -f mariadb-master-sc.yaml
storageclass.storage.k8s.io/mariadb-master-sc created
[vagrant@kubemaster mariadb-master]$ kb describe sc mariadb-master-sc
Name: mariadb-master-sc
IsDefaultClass: No
Annotations: <none>
Provisioner: kubernetes.io/no-provisioner
Parameters: <none>
AllowVolumeExpansion: <unset>
MountOptions: <none>
ReclaimPolicy: Delete
VolumeBindingMode: WaitForFirstConsumer
Events: <none>
[vagrant@kubemaster mariadb-master]$
[vagrant@kubemaster mariadb-master]$ vi mariadb-master-pv.yaml
apiVersion: v1
kind: PersistentVolume
metadata:
name: mariadb-master-disk
spec:
capacity:
storage: 1Gi
# volumeMode field requires BlockVolume Alpha feature gate to be enabled.
volumeMode: Filesystem
accessModes:
- ReadWriteOnce
persistentVolumeReclaimPolicy: Recycle
storageClassName: mariadb-master-sc
local:
path: /db1
nodeAffinity:
required:
nodeSelectorTerms:
- matchExpressions:
- key: kubernetes.io/hostname
operator: In
values:
- kubeworker1
[vagrant@kubemaster mariadb-master]$ kb create -f mariadb-master-pv.yaml
persistentvolume/mariadb-master-disk created
[vagrant@kubemaster mariadb-master]$
[vagrant@kubemaster mariadb-master]$ kb get pv
NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE
mariadb-master-disk 1Gi RWO Recycle Available mariadb-master-sc 5m57s
2) Create service for mariadb-master
[vagrant@kubemaster mariadb-master]$ vi mariadb-master-svc.yaml
apiVersion: v1
kind: Service
metadata:
name: mariadb-master
labels:
app: mariadb-master
spec:
ports:
- name: mariadb-master
port: 3306
clusterIP: None
selector:
app: mariadb-master
~
[vagrant@kubemaster mariadb-master]$ kb create -f mariadb-master-svc.yaml
service/mariadb-master created
[vagrant@kubemaster mariadb-master]$ kb get svc
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 5d7h
mariadb-master ClusterIP None <none> 3306/TCP 7s
[vagrant@kubemaster mariadb-master]$ kb describe svc mariadb-master
Name: mariadb-master
Namespace: default
Labels: app=mariadb-master
Annotations: <none>
Selector: app=mariadb-master
Type: ClusterIP
IP: None
Port: mariadb-master 3306/TCP
TargetPort: 3306/TCP
Endpoints: <none>
Session Affinity: None
Events: <none>
3) Creating configmap for maraidb-config cnf for master service.
[vagrant@kubemaster mariadb-master]$ vi configmap-mariadb-master.cnf
# MariaDB-specific config file.
# Read by /etc/mysql/my.cnf
#
[client]
# Default is Latin1, if you need UTF-8 set this (also in server section)
# default-character-set = utf8
#
[mysqld]
#master server for replication
log-bin
server_id=1
log-basename=master1
# * Character sets
# Default is Latin1, if you need UTF-8 set all this (also in client section)
character_set_server = utf8
collation_server = utf8_general_ci
#
[mysqld_safe]
skip_log_error
syslog
[mariadb]
# See https://mariadb.com/kb/en/how-to-enable-tokudb-in-mariadb/
# # for instructions how to enable TokuDB
# #
# # See https://mariadb.com/kb/en/tokudb-differences/ for differences
# # between TokuDB in MariaDB and TokuDB from http://www.tokutek.com/
#
# #plugin-load-add=ha_tokudb.so
[vagrant@kubemaster mariadb-master]$ kb create configmap mariadb-master.cf --from-file=configmap-mariadb-master.cnf
configmap/mariadb-master.cf created
[vagrant@kubemaster mariadb-master]$ kb describe configmap mariadb-master.cf
Name: mariadb-master.cf
Namespace: default
Labels: <none>
Annotations: <none>
Data
====
configmap-mariadb-master.cnf:
----
# MariaDB-specific config file.
# Read by /etc/mysql/my.cnf
#
[client]
# Default is Latin1, if you need UTF-8 set this (also in server section)
# default-character-set = utf8
#
[mysqld]
#master server for replication
log-bin
server_id=1
log-basename=master1
# * Character sets
# Default is Latin1, if you need UTF-8 set all this (also in client section)
character_set_server = utf8
collation_server = utf8_general_ci
#
[mysqld_safe]
skip_log_error
syslog
[mariadb]
# See https://mariadb.com/kb/en/how-to-enable-tokudb-in-mariadb/
# # for instructions how to enable TokuDB
# #
# # See https://mariadb.com/kb/en/tokudb-differences/ for differences
# # between TokuDB in MariaDB and TokuDB from http://www.tokutek.com/
#
# #plugin-load-add=ha_tokudb.so
Events: <none>
4) Create Stateful mariadb-master
[vagrant@kubemaster mariadb-master]$ vi kustomization.yaml
ecretGenerator:
- name: mariadb-pass
literals:
- password=StrongPass$^^$
- name: rep-user
literals:
- rep-user=rep-user
- name: rep-password
literals:
- rep-password=Good$^Password!
resources:
- mariadb-master.yaml
[vagrant@kubemaster mariadb-master]$ vi mariadb-master.yaml
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: mariadb-master
spec:
selector:
matchLabels:
app: mariadb-master
serviceName: mariadb-master
replicas: 1
template:
metadata:
labels:
app: mariadb-master
spec:
containers:
- name: mariadb-master
image: ohyoungjooung2/mariadb:10.1.14-master
env:
- name: MYSQL_ROOT_PASSWORD
valueFrom:
secretKeyRef:
name: mariadb-pass
key: password
- name: MYSQL_REPLICATION_USER
valueFrom:
secretKeyRef:
name: rep-user
key: rep-user
- name: MYSQL_REPLICATION_PASSWORD
valueFrom:
secretKeyRef:
name: rep-password
key: rep-password
ports:
- name: mariadb-master
containerPort: 3306
volumeMounts:
- name: mariadb-master-st
mountPath: /var/lib/mysql
subPath: mariadb
- name: mariadb-master-cnf
mountPath: /etc/mysql/conf.d
resources:
requests:
cpu: 500m
memory: 1Gi
livenessProbe:
exec:
command: ["mysqladmin", "ping"]
initialDelaySeconds: 30
periodSeconds: 10
timeoutSeconds: 5
readinessProbe:
exec:
# Check we can execute queries over TCP (skip-networking is off).
command: ["mysql", "-h", "localhost","-pStrongPass$^^$","-e", "SELECT 1"]
initialDelaySeconds: 5
periodSeconds: 2
timeoutSeconds: 1
volumes:
- name: mariadb-master-cnf
configMap:
name: mariadb-master.cf
volumeClaimTemplates:
- metadata:
name: mariadb-master-st
spec:
accessModes: ["ReadWriteOnce"]
storageClassName: "mariadb-master-sc"
resources:
requests:
storage: 1Gi
[vagrant@kubemaster mariadb-master]$ kb apply -k .
secret/mariadb-pass-hmt2hb8m6g created
secret/rep-password-8bhkm96dbt created
secret/rep-user-bttd79k4t8 created
statefulset.apps/mariadb-master created
[vagrant@kubemaster mariadb-master]$ kb get po
NAME READY STATUS RESTARTS AGE
mariadb-master-0 0/1 ContainerCreating 0 3s
[vagrant@kubemaster mariadb-master]$ kb get po
NAME READY STATUS RESTARTS AGE
mariadb-master-0 1/1 Running 0 92s
[vagrant@kubemaster mariadb-master]$ kb describe po mariadb-master-0
Name: mariadb-master-0
Namespace: default
Priority: 0
Node: kubeworker1/192.168.121.6
Start Time: Thu, 04 Jul 2019 17:43:46 +0000
Labels: app=mariadb-master
controller-revision-hash=mariadb-master-5cdc8d5bfc
statefulset.kubernetes.io/pod-name=mariadb-master-0
Annotations: <none>
Status: Running
IP: 10.244.2.3
Controlled By: StatefulSet/mariadb-master
Containers:
mariadb-master:
Container ID: docker://854f12654c97984d383663a1ad38e691f11cee66c3ae25a2b2ac86f0b95df996
Image: ohyoungjooung2/mariadb:10.1.14-master
Image ID: docker-pullable://ohyoungjooung2/mariadb@sha256:1a1011fb68a7d94c30aa60a74749c439041522813ab1649c3033cb846946cc75
Port: 3306/TCP
Host Port: 0/TCP
State: Running
Started: Thu, 04 Jul 2019 17:43:47 +0000
Ready: True
Restart Count: 0
Requests:
cpu: 500m
memory: 1Gi
Liveness: exec [mysqladmin ping] delay=30s timeout=5s period=10s #success=1 #failure=3
Readiness: exec [mysql -h localhost -pStrongPass$^^$ -e SELECT 1] delay=5s timeout=1s period=2s #success=1 #failure=3
Environment:
MYSQL_ROOT_PASSWORD: <set to the key 'password' in secret 'mariadb-pass-hmt2hb8m6g'> Optional: false
MYSQL_REPLICATION_USER: <set to the key 'rep-user' in secret 'rep-user-bttd79k4t8'> Optional: false
MYSQL_REPLICATION_PASSWORD: <set to the key 'rep-password' in secret 'rep-password-8bhkm96dbt'> Optional: false
Mounts:
/etc/mysql/conf.d from mariadb-master-cnf (rw)
/var/lib/mysql from mariadb-master-st (rw,path="mariadb")
/var/run/secrets/kubernetes.io/serviceaccount from default-token-7pn58 (ro)
Conditions:
Type Status
Initialized True
Ready True
ContainersReady True
PodScheduled True
Volumes:
mariadb-master-st:
Type: PersistentVolumeClaim (a reference to a PersistentVolumeClaim in the same namespace)
ClaimName: mariadb-master-st-mariadb-master-0
ReadOnly: false
mariadb-master-cnf:
Type: ConfigMap (a volume populated by a ConfigMap)
Name: mariadb-master.cf
Optional: false
default-token-7pn58:
Type: Secret (a volume populated by a Secret)
SecretName: default-token-7pn58
Optional: false
QoS Class: Burstable
Node-Selectors: <none>
Tolerations: node.kubernetes.io/not-ready:NoExecute for 300s
node.kubernetes.io/unreachable:NoExecute for 300s
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Scheduled 32s default-scheduler Successfully assigned default/mariadb-master-0 to kubeworker1
Normal Pulled 31s kubelet, kubeworker1 Container image "ohyoungjooung2/mariadb:10.1.14-master" already present on machine
Normal Created 30s kubelet, kubeworker1 Created container mariadb-master
Normal Started 30s kubelet, kubeworker1 Started container mariadb-master
5) mysql connection test.
root@mariadb-master-0:/# mysql -u root -p -h localhost
Enter password:
Welcome to the MariaDB monitor. Commands end with ; or \g.
Your MariaDB connection id is 514
Server version: 10.1.14-MariaDB-1~jessie mariadb.org binary distribution
Copyright (c) 2000, 2016, Oracle, MariaDB Corporation Ab and others.
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
MariaDB [(none)]> SHOW MASTER STATUS\G;
*************************** 1. row ***************************
File: master1-bin.000006
Position: 329
Binlog_Do_DB:
Binlog_Ignore_DB:
1 row in set (0.00 sec)
ERROR: No query specified
MariaDB [(none)]> SELECT user,host,password from mysql.user;
+----------+------+-------------------------------------------+
| user | host | password |
+----------+------+-------------------------------------------+
| root | % | *D674B2C0175FB3763DD6952371BE0DA43805D876 |
| rep-user | % | *7AA91BE9EE4FF62D8C576A7CA3B4CCBFD3F9EB19 |
+----------+------+-------------------------------------------+
2 rows in set (0.01 sec)
Conclusion)
In this small article, I introduce how to provisioing mariadb master server using pod.
In real production, database server's installation would be better on bare metal server, but this solution could be considered if someone like
all pod and container native.
Next). Two pod that replicate this master server(mariadb of course).
If you know more about statefulset, please visit https://kubernetes.io/docs/concepts/workloads/controllers/statefulset/
Thursday, July 4, 2019
Tuesday, July 2, 2019
UBUNTU18+KVM+VAGRANT+KUBERNETES 5)Install ansible and disk partition multi disks
This article shows how to install ansible and using playbook(.yaml) on ubuntu18 and centos7 kvm guest os to create directory,filesystem and mounted(/etc/fstab).
Below Youtube also explains..^^
https://www.youtube.com/watch?v=dVPiZi2daNA&feature=youtu.be
oyj@oyj-X555QG:~/ansible/pb$ which pip
/usr/bin/pip
oyj@oyj-X555QG:~/ansible/pb$ pip install ansible --user
Collecting ansible
Collecting jinja2 (from ansible)
Using cached https://files.pythonhosted.org/packages/1d/e7/fd8b501e7a6dfe492a433deb7b9d833d39ca74916fa8bc63dd1a4947a671/Jinja2-2.10.1-py2.py3-none-any.whl
Collecting PyYAML (from ansible)
Collecting cryptography (from ansible)
Using cached https://files.pythonhosted.org/packages/e6/68/50698ce24c61db7d44d93a5043c621a0ca7839d4ef9dff913e6ab465fc92/cryptography-2.7-cp27-cp27mu-manylinux1_x86_64.whl
Collecting MarkupSafe>=0.23 (from jinja2->ansible)
Using cached https://files.pythonhosted.org/packages/fb/40/f3adb7cf24a8012813c5edb20329eb22d5d8e2a0ecf73d21d6b85865da11/MarkupSafe-1.1.1-cp27-cp27mu-manylinux1_x86_64.whl
Collecting asn1crypto>=0.21.0 (from cryptography->ansible)
Using cached https://files.pythonhosted.org/packages/ea/cd/35485615f45f30a510576f1a56d1e0a7ad7bd8ab5ed7cdc600ef7cd06222/asn1crypto-0.24.0-py2.py3-none-any.whl
Collecting enum34; python_version < "3" (from cryptography->ansible)
Using cached https://files.pythonhosted.org/packages/c5/db/e56e6b4bbac7c4a06de1c50de6fe1ef3810018ae11732a50f15f62c7d050/enum34-1.1.6-py2-none-any.whl
Collecting ipaddress; python_version < "3" (from cryptography->ansible)
Using cached https://files.pythonhosted.org/packages/fc/d0/7fc3a811e011d4b388be48a0e381db8d990042df54aa4ef4599a31d39853/ipaddress-1.0.22-py2.py3-none-any.whl
Collecting cffi!=1.11.3,>=1.8 (from cryptography->ansible)
Using cached https://files.pythonhosted.org/packages/8d/e9/0c8afd1579e5cf7bc0f06fbcd7cdb954cbc0baadd505973949a99337da1c/cffi-1.12.3-cp27-cp27mu-manylinux1_x86_64.whl
Collecting six>=1.4.1 (from cryptography->ansible)
Using cached https://files.pythonhosted.org/packages/73/fb/00a976f728d0d1fecfe898238ce23f502a721c0ac0ecfedb80e0d88c64e9/six-1.12.0-py2.py3-none-any.whl
Collecting pycparser (from cffi!=1.11.3,>=1.8->cryptography->ansible)
Installing collected packages: MarkupSafe, jinja2, PyYAML, asn1crypto, enum34, ipaddress, pycparser, cffi, six, cryptography, ansible
Successfully installed MarkupSafe-1.1.1 PyYAML-5.1.1 ansible-2.8.1 asn1crypto-0.24.0 cffi-1.12.3 cryptography-2.7 enum34-1.1.6 ipaddress-1.0.22 jinja2-2.10.1 pycparser-2.19 six-1.12.0
Segmentation fault (core dumped)
oyj@oyj-X555QG:~/ansible/pb$ echo -e "\e[34m Some package might be not workingnow, but ok I think \e[00m "
oyj@oyj-X555QG:~/ansible/pb$ echo -e "\e[34m Some package might be not workingnow, but ok I think \e[00m "
Some package might be not workingnow, but ok I think
oyj@oyj-X555QG:~/ansible/pb$ cat ~/.bashrc | grep -i "ansible"
export ANSIBLE_HOME=$HOME/ansible
export ANSIBLE_CONFIG=$HOME/ansible/ansible.cfg
export ANSIBLE_HOST_KEY_CHECKING=false
oyj@oyj-X555QG:~/ansible/pb$ echo $PATH | grep ".local"
/home/oyj/.local/bin:/home/oyj/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin
oyj@oyj-X555QG:~/ansible/pb$ cat ~/ansible/ansible.cfg | grep hosts
inventory = /home/oyj/ansible/hosts
oyj@oyj-X555QG:~/ansible/pb$ cat ~/ansible/hosts
[kuber]
10.1.0.2
10.1.0.3
10.1.0.4
10.1.0.5
[kuber:vars]
ansible_private_key_file=~/INSTALL/u18kvk8s/k8s/id_rsa
ansible_ssh_user=vagrant
oyj@oyj-X555QG:~/ansible/pb$ ansible kuber -m ping
oyj@oyj-X555QG:~/ansible/pb$ ansible kuber -m ping
^[[A10.1.0.3 | SUCCESS => {
"ansible_facts": {
"discovered_interpreter_python": "/usr/bin/python"
},
"changed": false,
"ping": "pong"
}
10.1.0.5 | SUCCESS => {
"ansible_facts": {
"discovered_interpreter_python": "/usr/bin/python"
},
"changed": false,
"ping": "pong"
}
10.1.0.4 | SUCCESS => {
"ansible_facts": {
"discovered_interpreter_python": "/usr/bin/python"
},
"changed": false,
"ping": "pong"
}
10.1.0.2 | SUCCESS => {
"ansible_facts": {
"discovered_interpreter_python": "/usr/bin/python"
},
"changed": false,
"ping": "pong"
oyj@oyj-X555QG:~/ansible/pb$ echo -e "\e[34m Seems like it is good! \e[00m "
Seems like it is good!
oyj@oyj-X555QG:~/ansible/pb$ echo -e "\e[34m Now I will add disks and add filesystem onto k8s nodes \e[00m "
oyj@oyj-X555QG:~/ansible/pb$ echo -e "\e[34m For 3 nodes and 9 disks with onne command, it still needs some yaml \e[00m "
Now I will add disks and add filesystem onto k8s nodes
oyj@oyj-X555QG:~/ansible/pb$ cat INSTALL/u18kvk8s/k8s/Vagrantfile
# -*- mode: ruby -*-
# vi: set ft=ruby :
# Vagrantfile API/syntax version. Don't touch unless you know what you're doing!
VAGRANTFILE_API_VERSION = "2"
Vagrant.configure(VAGRANTFILE_API_VERSION) do |config|
#config.vm.synced_folder ".","/vagrant",type:"basic"
config.vm.define "kubemaster", primary: true do |kubemaster|
kubemaster.vm.box = "centos/7"
kubemaster.vm.provision "shell", path: "check_key.sh"
kubemaster.vm.provision "shell", :path => "swapoff.sh"
kubemaster.vm.provision "shell", :path => "k8s_docker_install.sh"
kubemaster.vm.provision "shell", :path => "k8s_install.sh"
kubemaster.vm.provision "shell", :path => "kubeadm-flannel.sh"
kubemaster.vm.network "private_network", ip:"10.1.0.2"
kubemaster.vm.host_name = "kubemaster"
kubemaster.vm.provider :libvirt do |lv|
lv.cpus = 2
lv.memory = 2048
end
end
config.vm.define "kubeworker1" do |kubeworker1|
kubeworker1.vm.box = "centos/7"
kubeworker1.vm.provision "shell", path: "check_key.sh"
kubeworker1.vm.provision "file", source: "id_rsa",destination: "/home/vagrant/.ssh/id_rsa"
kubeworker1.vm.provision "shell", :path => "swapoff.sh"
kubeworker1.vm.provision "shell", :path => "k8s_docker_install.sh"
kubeworker1.vm.provision "shell", :path => "k8s_install.sh"
kubeworker1.vm.provision "shell", :path => "admjoin.sh"
kubeworker1.vm.network "private_network", ip:"10.1.0.3"
kubeworker1.vm.host_name = "kubeworker1"
kubeworker1.vm.provider :libvirt do |lv|
lv.memory = 2048
lv.storage :file, :size => '1G', :type => 'raw'
lv.storage :file, :size => '1G', :type => 'raw'
lv.storage :file, :size => '1G', :type => 'raw'
end
end
config.vm.define "kubeworker2" do |kubeworker2|
kubeworker2.vm.box = "centos/7"
kubeworker2.vm.provision "shell", path: "check_key.sh"
kubeworker2.vm.provision "file", source: "id_rsa",destination: "/home/vagrant/.ssh/id_rsa"
kubeworker2.vm.provision "shell", :path => "swapoff.sh"
kubeworker2.vm.provision "shell", :path => "k8s_docker_install.sh"
kubeworker2.vm.provision "shell", :path => "k8s_install.sh"
kubeworker2.vm.provision "shell", :path => "admjoin.sh"
kubeworker2.vm.network "private_network", ip:"10.1.0.4"
kubeworker2.vm.host_name = "kubeworker2"
kubeworker2.vm.provider :libvirt do |lv|
lv.memory = 2048
lv.storage :file, :size => '1G', :type => 'raw'
lv.storage :file, :size => '1G', :type => 'raw'
lv.storage :file, :size => '1G', :type => 'raw'
end
end
config.vm.define "kubeworker3" do |kubeworker3|
kubeworker3.vm.box = "centos/7"
kubeworker3.vm.provision "shell", path: "check_key.sh"
kubeworker3.vm.provision "file", source: "id_rsa",destination: "/home/vagrant/.ssh/id_rsa"
kubeworker3.vm.provision "shell", :path => "swapoff.sh"
kubeworker3.vm.provision "shell", :path => "k8s_docker_install.sh"
kubeworker3.vm.provision "shell", :path => "k8s_install.sh"
kubeworker3.vm.provision "shell", :path => "admjoin.sh"
kubeworker3.vm.network "private_network", ip:"10.1.0.5"
kubeworker3.vm.host_name = "kubeworker3"
kubeworker3.vm.provider :libvirt do |lv|
lv.memory = 2048
lv.storage :file, :size => '1G', :type => 'raw'
lv.storage :file, :size => '1G', :type => 'raw'
lv.storage :file, :size => '1G', :type => 'raw'
end
end
end
oyj@oyj-X555QG:~/ansible/pb$ cat disk_ext4.yaml
---
#Below is k8s workder nodes that has 3 disks
- hosts: 10.1.0.5,10.1.0.4,10.1.0.3
become: yes
tasks:
# Create file system on each nodes' local disks
- name: Create ext4 file systems
filesystem:
fstype: ext4
dev: "{{ item }}"
force: yes
with_items:
- /dev/vdb
- /dev/vdc
- /dev/vdd
- name: Create mount dir for /dev/vdxx
file:
path: "{{ item }}"
state: directory
owner: root
group: root
with_items:
- /db1
- /db2
- /web
- name: Mount each dev to each dir
mount:
path: "{{item.tar}}"
src: "{{item.src}}"
fstype: ext4
state: mounted
with_items:
- { src: '/dev/vdb', tar: '/db1' }
- { src: '/dev/vdc', tar: '/db2' }
- { src: '/dev/vdd', tar: '/web' }
oyj@oyj-X555QG:~/ansible/pb$ ansible-playbook disk_ext4.yaml
PLAY [10.1.0.5,10.1.0.4,10.1.0.3] ***********************************************************************
TASK [Gathering Facts] **********************************************************************************
ok: [10.1.0.3]
ok: [10.1.0.5]
ok: [10.1.0.4]
TASK [Create ext4 file systems] *************************************************************************
changed: [10.1.0.5] => (item=/dev/vdb)
changed: [10.1.0.3] => (item=/dev/vdb)
changed: [10.1.0.4] => (item=/dev/vdb)
changed: [10.1.0.5] => (item=/dev/vdc)
changed: [10.1.0.3] => (item=/dev/vdc)
changed: [10.1.0.4] => (item=/dev/vdc)
changed: [10.1.0.5] => (item=/dev/vdd)
changed: [10.1.0.4] => (item=/dev/vdd)
changed: [10.1.0.3] => (item=/dev/vdd)
TASK [Create mount dir for /dev/vdxx] *******************************************************************
ok: [10.1.0.5] => (item=/db1)
ok: [10.1.0.4] => (item=/db1)
ok: [10.1.0.3] => (item=/db1)
ok: [10.1.0.3] => (item=/db2)
ok: [10.1.0.5] => (item=/db2)
ok: [10.1.0.4] => (item=/db2)
ok: [10.1.0.3] => (item=/web)
ok: [10.1.0.5] => (item=/web)
ok: [10.1.0.4] => (item=/web)
TASK [Mount each dev to each dir] ***********************************************************************
changed: [10.1.0.4] => (item={u'src': u'/dev/vdb', u'tar': u'/db1'})
changed: [10.1.0.3] => (item={u'src': u'/dev/vdb', u'tar': u'/db1'})
changed: [10.1.0.5] => (item={u'src': u'/dev/vdb', u'tar': u'/db1'})
changed: [10.1.0.4] => (item={u'src': u'/dev/vdc', u'tar': u'/db2'})
changed: [10.1.0.3] => (item={u'src': u'/dev/vdc', u'tar': u'/db2'})
changed: [10.1.0.5] => (item={u'src': u'/dev/vdc', u'tar': u'/db2'})
changed: [10.1.0.4] => (item={u'src': u'/dev/vdd', u'tar': u'/web'})
changed: [10.1.0.3] => (item={u'src': u'/dev/vdd', u'tar': u'/web'})
changed: [10.1.0.5] => (item={u'src': u'/dev/vdd', u'tar': u'/web'})
PLAY RECAP **********************************************************************************************
10.1.0.3 : ok=4 changed=2 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
10.1.0.4 : ok=4 changed=2 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
10.1.0.5 : ok=4 changed=2 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
oyj@oyj-X555QG:~/ansible/pb$ ansible kuber -m shell -a 'df -h | grep /web' -b
10.1.0.5 | CHANGED | rc=0 >>
/dev/vdd 976M 2.6M 907M 1% /web
10.1.0.3 | CHANGED | rc=0 >>
/dev/vdd 976M 2.6M 907M 1% /web
10.1.0.4 | CHANGED | rc=0 >>
/dev/vdd 976M 2.6M 907M 1% /web
10.1.0.2 | FAILED | rc=1 >>
non-zero return code
oyj@oyj-X555QG:~/ansible/pb$ ansible kuber -m shell -a 'df -h | grep /db' -b
10.1.0.4 | CHANGED | rc=0 >>
/dev/vdb 976M 2.6M 907M 1% /db1
/dev/vdc 976M 2.6M 907M 1% /db2
10.1.0.3 | CHANGED | rc=0 >>
/dev/vdb 976M 2.6M 907M 1% /db1
/dev/vdc 976M 2.6M 907M 1% /db2
10.1.0.5 | CHANGED | rc=0 >>
/dev/vdb 976M 2.6M 907M 1% /db1
/dev/vdc 976M 2.6M 907M 1% /db2
10.1.0.2 | CHANGED | rc=0 >>
overlay 40G 4.2G 36G 11% /var/lib/docker/overlay2/db364a3d78c278920f9564755bf51cbee21cdf00951326cff7d4d0907a621b0b/merged
Saturday, June 29, 2019
PYTHON3-AVG-EXCEPT-FIRST-AND-LAST-DATA
oyj@oyj-X555QG:~/pycode$ cat avg_middle.py
#!/usr/bin/env python3
from statistics import mean
#When you want to get avg value from list except first and last value.
#More specific example for python3 cookbook.3rd edition
def drop_first_last_avg(grades):
first,*middle,last = grades
return mean(middle)
data=[10,80,80,80,80,100]
print(drop_first_last_avg(data))
oyj@oyj-X555QG:~/pycode$ python3 avg_middle.py
80
oyj@oyj-X555QG:~/pycode$ chmod 700 avg_middle.py
oyj@oyj-X555QG:~/pycode$ ./avg_middle.py
80
#!/usr/bin/env python3
from statistics import mean
#When you want to get avg value from list except first and last value.
#More specific example for python3 cookbook.3rd edition
def drop_first_last_avg(grades):
first,*middle,last = grades
return mean(middle)
data=[10,80,80,80,80,100]
print(drop_first_last_avg(data))
oyj@oyj-X555QG:~/pycode$ python3 avg_middle.py
80
oyj@oyj-X555QG:~/pycode$ chmod 700 avg_middle.py
oyj@oyj-X555QG:~/pycode$ ./avg_middle.py
80
Thursday, September 1, 2016
Using ruby array insert
##When I have to insert numbers that are omitted between numbers###
##For example, the file contents are as follows.
00:data
04:data
05:data
06:data
07:data
08:data
09:data
10:data
13:data
14:data
15:data
16:data
19:data
I need to insert 01,02,03 between 00 and 04. And I also need to insert 11,12.
On linux box, I may have to use awk sed or perl. But for me ruby is much easier.
The code is as follows.
[root@redhat2 ~]# cat t.rb
#!/usr/bin/env ruby
#Function to less than 9 insert method differently
def more_than_9(a,n)
if n < 9
a.insert(n,"0#{n}:data")
else
a.insert(n,"#{n}:data")
end
end
#File open and into array
a=File.new("./test.csv").read.split("\n")
s=a.size
i=0
while i < (s-1)
if (a[i+1].split(":")[0].to_i - a[i].split(":")[0].to_i) > 1
nx=a[i+1].split(":")[0].to_i
while i+1 < nx
n=i+1
more_than_9(a,n)
i=n
end
end
i=i+1
end
#Puts results
puts a
#So, the result of executing this script is as follows.
[root@redhat2 ~]# ruby t.rb
00:data
01:data
02:data
03:data
04:data
05:data
06:data
07:data
08:data
09:data
10:data
11:data
12:data
13:data
14:data
15:data
16:data
19:data
# #################################3
##For example, the file contents are as follows.
00:data
04:data
05:data
06:data
07:data
08:data
09:data
10:data
13:data
14:data
15:data
16:data
19:data
I need to insert 01,02,03 between 00 and 04. And I also need to insert 11,12.
On linux box, I may have to use awk sed or perl. But for me ruby is much easier.
The code is as follows.
[root@redhat2 ~]# cat t.rb
#!/usr/bin/env ruby
#Function to less than 9 insert method differently
def more_than_9(a,n)
if n < 9
a.insert(n,"0#{n}:data")
else
a.insert(n,"#{n}:data")
end
end
#File open and into array
a=File.new("./test.csv").read.split("\n")
s=a.size
i=0
while i < (s-1)
if (a[i+1].split(":")[0].to_i - a[i].split(":")[0].to_i) > 1
nx=a[i+1].split(":")[0].to_i
while i+1 < nx
n=i+1
more_than_9(a,n)
i=n
end
end
i=i+1
end
#Puts results
puts a
#So, the result of executing this script is as follows.
[root@redhat2 ~]# ruby t.rb
00:data
01:data
02:data
03:data
04:data
05:data
06:data
07:data
08:data
09:data
10:data
11:data
12:data
13:data
14:data
15:data
16:data
19:data
# #################################3
Finding same file size in a current directory(ruby)-dirty script
#!/usr/bin/env ruby
#This is dirty ruby script but function well though.
#More with " https://github.com/ohyoungjooung2/ruby_scripts/blob/master/same_file.rb"
#^^; Have fun.
files=`find . -type f | xargs stat -c "%s %n"`
s=files.split("\n")
#p s
sk=[]
sv=[]
i=0
s_size=s.size
while i < s_size
sk<<s[i].split(" ")[1]
sv<<s[i].split(" ")[0]
i+=1
end
# files name as a array sk
#p sk
#puts ""
# files size as a array sv
#p sv
#Creating hash
j=0
h={}
#puts "sv size is #{sv.size}"
#puts "s_size is #{s_size}"
while j < sv.size
#puts j
h[sk[j]]=sv[j]
j+=1
end
#Putting h hash
#puts h
z=0
#Getting uniq values of each hash value
u=h.values.uniq
while z < h.size
# puts z
same_files=h.select { |k,v| v==u[z] }
if same_files.size >= 2
fs=0
while fs < same_files.size
puts "same_file size with #{u[z]} bytes are #{same_files.keys[fs]}"
fs+=1
end
end
z+=1
end
young@ubuntu-16:~/test$ ruby -v
ruby 2.3.1p112 (2016-04-26 revision 54768) [x86_64-linux]
young@ubuntu-16:~/test$ chmod 700 same_file.rb
young@ubuntu-16:~/test$ ./same_file.rb
same_file size with 8941 bytes are ./1
same_file size with 8941 bytes are ./2
same_file size with 548 bytes are ./apparmor.d/abstractions/ibus
same_file size with 548 bytes are ./f.sh
same_file size with 1748 bytes are ./tttt/iiii
same_file size with 1748 bytes are ./tttt/inputrc
same_file size with 1748 bytes are ./inputrc
same_file size with 1654 bytes are ./tttt/pass
same_file size with 1654 bytes are ./tttt/passwd
same_file size with 1654 bytes are ./pass
same_file size with 1654 bytes are ./passwd
same_file size with 1654 bytes are ./ssap
same_file size with 26 bytes are ./tttt/iss
same_file size with 26 bytes are ./tttt/issue2
same_file size with 26 bytes are ./issue
same_file size with 364 bytes are ./tttt/rrrr
same_file size with 364 bytes are ./rc.local
same_file size with 738 bytes are ./same_file.rb
same_file size with 738 bytes are ./groff/man.local
#This is dirty ruby script but function well though.
#More with " https://github.com/ohyoungjooung2/ruby_scripts/blob/master/same_file.rb"
#^^; Have fun.
files=`find . -type f | xargs stat -c "%s %n"`
s=files.split("\n")
#p s
sk=[]
sv=[]
i=0
s_size=s.size
while i < s_size
sk<<s[i].split(" ")[1]
sv<<s[i].split(" ")[0]
i+=1
end
# files name as a array sk
#p sk
#puts ""
# files size as a array sv
#p sv
#Creating hash
j=0
h={}
#puts "sv size is #{sv.size}"
#puts "s_size is #{s_size}"
while j < sv.size
#puts j
h[sk[j]]=sv[j]
j+=1
end
#Putting h hash
#puts h
z=0
#Getting uniq values of each hash value
u=h.values.uniq
while z < h.size
# puts z
same_files=h.select { |k,v| v==u[z] }
if same_files.size >= 2
fs=0
while fs < same_files.size
puts "same_file size with #{u[z]} bytes are #{same_files.keys[fs]}"
fs+=1
end
end
z+=1
end
young@ubuntu-16:~/test$ ruby -v
ruby 2.3.1p112 (2016-04-26 revision 54768) [x86_64-linux]
young@ubuntu-16:~/test$ chmod 700 same_file.rb
young@ubuntu-16:~/test$ ./same_file.rb
same_file size with 8941 bytes are ./1
same_file size with 8941 bytes are ./2
same_file size with 548 bytes are ./apparmor.d/abstractions/ibus
same_file size with 548 bytes are ./f.sh
same_file size with 1748 bytes are ./tttt/iiii
same_file size with 1748 bytes are ./tttt/inputrc
same_file size with 1748 bytes are ./inputrc
same_file size with 1654 bytes are ./tttt/pass
same_file size with 1654 bytes are ./tttt/passwd
same_file size with 1654 bytes are ./pass
same_file size with 1654 bytes are ./passwd
same_file size with 1654 bytes are ./ssap
same_file size with 26 bytes are ./tttt/iss
same_file size with 26 bytes are ./tttt/issue2
same_file size with 26 bytes are ./issue
same_file size with 364 bytes are ./tttt/rrrr
same_file size with 364 bytes are ./rc.local
same_file size with 738 bytes are ./same_file.rb
same_file size with 738 bytes are ./groff/man.local
Monday, August 29, 2016
Finding same file size in a directory
young@ubuntu-16:~/test$ cat find_same_size.sh
#!/usr/bin/env bash
#set -x
#This is small script can find same size of files.
find_same_size(){
if [[ -z $1 || ! -d $1 ]]
then
echo "Usage $0 directory_name" ;
exit $?
else
dir_name=$1;
echo "current directory is $1"
for i in $(find $dir_name -type f); do
ls -fl $i
done | awk '{f=""
if(NF>9)for(i=9;i<=NF;i++)f=f?f" "$i:$i; else f=$9;
if(a[$5]){ a[$5]=a[$5]"\n"f; b[$5]++;} else a[$5]=f} END{for(x in b)print a[x] }' | xargs stat -c "%s %n" #For just list files
fi
}
find_same_size $1
#Example usage
young@ubuntu-16:~/test$ bash find_same_size.sh tttt/
current directory is tttt/
26 tttt/iss
26 tttt/issue2
1654 tttt/pass
1654 tttt/passwd
#We can delete duplicated files by using like below.
#First delete line that starts with alphabet. Only find the line that begins with numeric. and
#then with md5sum command if the files are really same or not
young@ubuntu-16:~/test$ bash find_same_size.sh tttt/ | awk '{ if($1 !~ /^([[:alpha:]])+/) print $2}' | xargs md5sum
1e7672cbc2f76c3e9daad0e290e711b9 tttt/iss
1e7672cbc2f76c3e9daad0e290e711b9 tttt/issue2
cc44dec6bfd51d296c89d55e7c38b933 tttt/pass
cc44dec6bfd51d296c89d55e7c38b933 tttt/passwd
#-w32 means 32bytes; -d deletes duplicated ones that created most recently among same md5sums. Finally with xargs we can do rm -vf
young@ubuntu-16:~/test$ bash find_same_size.sh tttt/ | awk '{ if($1 !~ /^([[:alpha:]])+/) print $2}' | xargs md5sum | uniq -w32 -d | xargs rm -vf
removed 'tttt/iss'
removed 'tttt/pass'
#!/usr/bin/env bash
#set -x
#This is small script can find same size of files.
find_same_size(){
if [[ -z $1 || ! -d $1 ]]
then
echo "Usage $0 directory_name" ;
exit $?
else
dir_name=$1;
echo "current directory is $1"
for i in $(find $dir_name -type f); do
ls -fl $i
done | awk '{f=""
if(NF>9)for(i=9;i<=NF;i++)f=f?f" "$i:$i; else f=$9;
if(a[$5]){ a[$5]=a[$5]"\n"f; b[$5]++;} else a[$5]=f} END{for(x in b)print a[x] }' | xargs stat -c "%s %n" #For just list files
fi
}
find_same_size $1
#Example usage
young@ubuntu-16:~/test$ bash find_same_size.sh tttt/
current directory is tttt/
26 tttt/iss
26 tttt/issue2
1654 tttt/pass
1654 tttt/passwd
#We can delete duplicated files by using like below.
#First delete line that starts with alphabet. Only find the line that begins with numeric. and
#then with md5sum command if the files are really same or not
young@ubuntu-16:~/test$ bash find_same_size.sh tttt/ | awk '{ if($1 !~ /^([[:alpha:]])+/) print $2}' | xargs md5sum
1e7672cbc2f76c3e9daad0e290e711b9 tttt/iss
1e7672cbc2f76c3e9daad0e290e711b9 tttt/issue2
cc44dec6bfd51d296c89d55e7c38b933 tttt/pass
cc44dec6bfd51d296c89d55e7c38b933 tttt/passwd
#-w32 means 32bytes; -d deletes duplicated ones that created most recently among same md5sums. Finally with xargs we can do rm -vf
young@ubuntu-16:~/test$ bash find_same_size.sh tttt/ | awk '{ if($1 !~ /^([[:alpha:]])+/) print $2}' | xargs md5sum | uniq -w32 -d | xargs rm -vf
removed 'tttt/iss'
removed 'tttt/pass'
Wednesday, August 17, 2016
Bash-sum of each array-Debug on
#!/usr/bin/env bash
#This script is very simple example that sum each array numbers.
#debug on
set -x
var="15899"
echo "first is ${var:0:1}"
echo "second is ${var:1:1}"
echo "third is ${var:2:1}"
echo "fourth is ${var:3:1}"
echo "fifth is ${var:4:1}"
sum=0
start=0
while (( $start < ${#var} ))
do
sum=$(( $sum+${var:start++:1} ))
done
echo -e "Total sum is \e[;35m $sum \e[;0m"
#Just testing for fun.
young@ubuntu-16:~$ bash sum.sh
+ var=15899
+ echo 'first is 1'
first is 1
+ echo 'second is 5'
second is 5
+ echo 'third is 8'
third is 8
+ echo 'fourth is 9'
fourth is 9
+ echo 'fifth is 9'
fifth is 9
+ sum=0
+ start=0
+ (( 0 < 5 ))
+ sum=1
+ (( 1 < 5 ))
+ sum=6
+ (( 2 < 5 ))
+ sum=14
+ (( 3 < 5 ))
+ sum=23
+ (( 4 < 5 ))
+ sum=32
+ (( 5 < 5 ))
+ echo -e 'Total sum is \e[;35m 32 \e[;0m'
Total sum is 32
young@ubuntu-16:~$
#This script is very simple example that sum each array numbers.
#debug on
set -x
var="15899"
echo "first is ${var:0:1}"
echo "second is ${var:1:1}"
echo "third is ${var:2:1}"
echo "fourth is ${var:3:1}"
echo "fifth is ${var:4:1}"
sum=0
start=0
while (( $start < ${#var} ))
do
sum=$(( $sum+${var:start++:1} ))
done
echo -e "Total sum is \e[;35m $sum \e[;0m"
#Just testing for fun.
young@ubuntu-16:~$ bash sum.sh
+ var=15899
+ echo 'first is 1'
first is 1
+ echo 'second is 5'
second is 5
+ echo 'third is 8'
third is 8
+ echo 'fourth is 9'
fourth is 9
+ echo 'fifth is 9'
fifth is 9
+ sum=0
+ start=0
+ (( 0 < 5 ))
+ sum=1
+ (( 1 < 5 ))
+ sum=6
+ (( 2 < 5 ))
+ sum=14
+ (( 3 < 5 ))
+ sum=23
+ (( 4 < 5 ))
+ sum=32
+ (( 5 < 5 ))
+ echo -e 'Total sum is \e[;35m 32 \e[;0m'
Total sum is 32
young@ubuntu-16:~$
Subscribe to:
Posts (Atom)