You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: LINUX.md
+19-12Lines changed: 19 additions & 12 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -113,6 +113,7 @@ Have you signed up to GitHub? If not, [do it right away](https://github.com/join
113
113

114
114
115
115
- Give it a name such as `Wagon Bootcamp` for example
116
+
- β It is important that your project **does not** have an organisation β
116
117
- Notice the `ID` automatically created for the project, e.g. `wagon-bootcamp-123456`
117
118
118
119

@@ -641,12 +642,14 @@ If you want to view your SSH configuration:
641
642
642
643
</details>
643
644
644
-
π¨π¨π¨ **The setup of your local machine is complete. All following terminal commands will be run from within your virtual machine terminal (via VS Code)** π¨π¨π¨
645
+
Congratulations, the setup of you local laptop is complete π We just need to fine tune your Virtual Machine πͺ
645
646
646
647
---
647
648
648
649
# Part 2: Configure your Virtual Machine 1
649
650
651
+
π¨π¨π¨ **All following terminal commands will be run from within your virtual machine terminal, via VS Code** π¨π¨π¨
652
+
650
653
651
654
## VM gcloud and Application Default Credentials
652
655
@@ -756,7 +759,7 @@ And the playbook should start running!
756
759
757
760
### What is the playbook installing?
758
761
759
-
This playbook is installing a few things, while the playbook is running, let's go through them:
762
+
While this playbook is running, lets go through what is being installed and configured:
760
763
- Updating system packages. Ubuntu uses the `APT` package manager.
761
764
- Changing the default shell from **bash** to **zsh**, a more customizable shell that is extensible and looks great!
762
765
- Installing the **Oh-My-ZSH** plugin for the **zsh** shell. We'll use it a bit later to add some quality of life plugins and extensions for `zsh`.
@@ -769,6 +772,8 @@ This playbook is installing a few things, while the playbook is running, let's g
769
772
770
773
The playbook is also running checks to see if things are installed or not. This is so you can safely re-run the playbook without any problems.
771
774
775
+
π‘ Curious about how Ansible playbooks are defined, take a look at the playbook being run at this [link here π](https://github.com/lewagon/data-engineering-setup/blob/main-automation/automation/vm-ansible-setup/playbooks/setup_vm_part1.yml)
776
+
772
777
773
778
## GitHub CLI
774
779
@@ -908,15 +913,9 @@ And the playbook should start running! If you're asked if you want VS Code to be
908
913
909
914
β If any errors occur, raise a ticket with a teacher. You can safely run the playbook again.
910
915
911
-
<details>
912
-
<summarymarkdown='span'>β Why two Ansible playbooks?</summary>
913
-
914
-
This second ansible playbook requires GitHub authorisation to fork the `lewagon/data-engineering-challenges` repository and it is also editing some of the Le Wagon recommended **dotfiles**. So we separated the process into two steps.
915
-
</details>
916
-
917
916
### What is the playbook installing?
918
917
919
-
This playbook is installing and configuring a things, while the playbook is running, let's go through them:
918
+
While the playbook is running, let's go through what is being installed and configured.
920
919
921
920
**Python and Poetry**
922
921
@@ -951,23 +950,31 @@ Ubuntu 22.04 has Python pre-installed, but not the version we're going to use. W
951
950
-**gcloud**: Auto-complete for the gcloud CLI tool
952
951
-**ssh-agent**: Saves your SSH password so you only have to enter it once per session.
953
952
-**direnv**: A tool to load `.envrc` files when you `cd` into a directory. Great for loading environment variables.
954
-
- Installing **Spark**: Spark is a distributed data processing framework
953
+
- Installing **Spark**: A distributed data processing framework
955
954
956
955
**Data Engineering Challenges Repository**
957
956
958
957
The challenges that you'll be working on throughout the bootcamp! The playbook is forking the **data-engineering-challenges** repository from **lewagon** to your own GitHub user. Then cloning that repository from your GitHub account down onto your Virtual Machine.
959
958
959
+
π‘ Curious about how Ansible playbooks are defined, take a look at the playbook being run at this [link here π](https://github.com/lewagon/data-engineering-setup/blob/main-automation/automation/vm-ansible-setup/playbooks/setup_vm_part2.yml)
960
+
961
+
<details>
962
+
<summarymarkdown='span'>β Why two Ansible playbooks?</summary>
963
+
964
+
This second ansible playbook requires GitHub authorisation to fork the `lewagon/data-engineering-challenges` repository and it is also editing some of the Le Wagon recommended **dotfiles**. So we separated the process into two steps.
965
+
</details>
966
+
960
967
### Restart Virtual Machine
961
968
962
-
Once the playbook has finished running, you need to completely shutdown your Virtual Machine so that some of the configuration updates (specifically **pyenv** and **Docker**).
969
+
Once the playbook has finished running, you need to completely shutdown your Virtual Machine so that some of the configuration updates properly (specifically **pyenv** and **Docker**).
963
970
964
971
To shutdown your VM, navigate to the GCP Compute Engine Instances [console page π](https://console.cloud.google.com/compute/instances).
965
972
966
973
Select your VM instance and click on the stop button:
967
974
968
975

969
976
970
-
Wait for a few minutes until the VM shows that it is completely off. You may need to refresh the page, the GCP Console doesn't dynamically update.
977
+
Wait for a few minutes until the VM shows that it is completely off. You may need to refresh the page, the GCP Console (web GUI) doesn't dynamically update.
971
978
972
979
When the VM is completely off, turn it on again by selecting the check box next to your instance and clicking **START/RESUME**. Give it a minute to turn on, then connect via VS Code.
Copy file name to clipboardExpand all lines: WINDOWS.md
+19-12Lines changed: 19 additions & 12 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -113,6 +113,7 @@ Have you signed up to GitHub? If not, [do it right away](https://github.com/join
113
113

114
114
115
115
- Give it a name such as `Wagon Bootcamp` for example
116
+
- β It is important that your project **does not** have an organisation β
116
117
- Notice the `ID` automatically created for the project, e.g. `wagon-bootcamp-123456`
117
118
118
119

@@ -613,12 +614,14 @@ If you want to view your SSH configuration:
613
614
614
615
</details>
615
616
616
-
π¨π¨π¨ **The setup of your local machine is complete. All following terminal commands will be run from within your virtual machine terminal (via VS Code)** π¨π¨π¨
617
+
Congratulations, the setup of you local laptop is complete π We just need to fine tune your Virtual Machine πͺ
617
618
618
619
---
619
620
620
621
# Part 2: Configure your Virtual Machine 1
621
622
623
+
π¨π¨π¨ **All following terminal commands will be run from within your virtual machine terminal, via VS Code** π¨π¨π¨
624
+
622
625
623
626
## VM gcloud and Application Default Credentials
624
627
@@ -728,7 +731,7 @@ And the playbook should start running!
728
731
729
732
### What is the playbook installing?
730
733
731
-
This playbook is installing a few things, while the playbook is running, let's go through them:
734
+
While this playbook is running, lets go through what is being installed and configured:
732
735
- Updating system packages. Ubuntu uses the `APT` package manager.
733
736
- Changing the default shell from **bash** to **zsh**, a more customizable shell that is extensible and looks great!
734
737
- Installing the **Oh-My-ZSH** plugin for the **zsh** shell. We'll use it a bit later to add some quality of life plugins and extensions for `zsh`.
@@ -741,6 +744,8 @@ This playbook is installing a few things, while the playbook is running, let's g
741
744
742
745
The playbook is also running checks to see if things are installed or not. This is so you can safely re-run the playbook without any problems.
743
746
747
+
π‘ Curious about how Ansible playbooks are defined, take a look at the playbook being run at this [link here π](https://github.com/lewagon/data-engineering-setup/blob/main-automation/automation/vm-ansible-setup/playbooks/setup_vm_part1.yml)
748
+
744
749
745
750
## GitHub CLI
746
751
@@ -880,15 +885,9 @@ And the playbook should start running! If you're asked if you want VS Code to be
880
885
881
886
β If any errors occur, raise a ticket with a teacher. You can safely run the playbook again.
882
887
883
-
<details>
884
-
<summarymarkdown='span'>β Why two Ansible playbooks?</summary>
885
-
886
-
This second ansible playbook requires GitHub authorisation to fork the `lewagon/data-engineering-challenges` repository and it is also editing some of the Le Wagon recommended **dotfiles**. So we separated the process into two steps.
887
-
</details>
888
-
889
888
### What is the playbook installing?
890
889
891
-
This playbook is installing and configuring a things, while the playbook is running, let's go through them:
890
+
While the playbook is running, let's go through what is being installed and configured.
892
891
893
892
**Python and Poetry**
894
893
@@ -923,23 +922,31 @@ Ubuntu 22.04 has Python pre-installed, but not the version we're going to use. W
923
922
-**gcloud**: Auto-complete for the gcloud CLI tool
924
923
-**ssh-agent**: Saves your SSH password so you only have to enter it once per session.
925
924
-**direnv**: A tool to load `.envrc` files when you `cd` into a directory. Great for loading environment variables.
926
-
- Installing **Spark**: Spark is a distributed data processing framework
925
+
- Installing **Spark**: A distributed data processing framework
927
926
928
927
**Data Engineering Challenges Repository**
929
928
930
929
The challenges that you'll be working on throughout the bootcamp! The playbook is forking the **data-engineering-challenges** repository from **lewagon** to your own GitHub user. Then cloning that repository from your GitHub account down onto your Virtual Machine.
931
930
931
+
π‘ Curious about how Ansible playbooks are defined, take a look at the playbook being run at this [link here π](https://github.com/lewagon/data-engineering-setup/blob/main-automation/automation/vm-ansible-setup/playbooks/setup_vm_part2.yml)
932
+
933
+
<details>
934
+
<summarymarkdown='span'>β Why two Ansible playbooks?</summary>
935
+
936
+
This second ansible playbook requires GitHub authorisation to fork the `lewagon/data-engineering-challenges` repository and it is also editing some of the Le Wagon recommended **dotfiles**. So we separated the process into two steps.
937
+
</details>
938
+
932
939
### Restart Virtual Machine
933
940
934
-
Once the playbook has finished running, you need to completely shutdown your Virtual Machine so that some of the configuration updates (specifically **pyenv** and **Docker**).
941
+
Once the playbook has finished running, you need to completely shutdown your Virtual Machine so that some of the configuration updates properly (specifically **pyenv** and **Docker**).
935
942
936
943
To shutdown your VM, navigate to the GCP Compute Engine Instances [console page π](https://console.cloud.google.com/compute/instances).
937
944
938
945
Select your VM instance and click on the stop button:
939
946
940
947

941
948
942
-
Wait for a few minutes until the VM shows that it is completely off. You may need to refresh the page, the GCP Console doesn't dynamically update.
949
+
Wait for a few minutes until the VM shows that it is completely off. You may need to refresh the page, the GCP Console (web GUI) doesn't dynamically update.
943
950
944
951
When the VM is completely off, turn it on again by selecting the check box next to your instance and clicking **START/RESUME**. Give it a minute to turn on, then connect via VS Code.
Copy file name to clipboardExpand all lines: _partials/ubuntu_ansible_part1.md
+3-1Lines changed: 3 additions & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -50,7 +50,7 @@ And the playbook should start running!
50
50
51
51
### What is the playbook installing?
52
52
53
-
This playbook is installing a few things, while the playbook is running, let's go through them:
53
+
While this playbook is running, lets go through what is being installed and configured:
54
54
- Updating system packages. Ubuntu uses the `APT` package manager.
55
55
- Changing the default shell from **bash** to **zsh**, a more customizable shell that is extensible and looks great!
56
56
- Installing the **Oh-My-ZSH** plugin for the **zsh** shell. We'll use it a bit later to add some quality of life plugins and extensions for `zsh`.
@@ -62,3 +62,5 @@ This playbook is installing a few things, while the playbook is running, let's g
62
62
- Install the **GitHub CLI**: the CLI tool that we'll use to interact with your GitHub account directly from the terminal.
63
63
64
64
The playbook is also running checks to see if things are installed or not. This is so you can safely re-run the playbook without any problems.
65
+
66
+
π‘ Curious about how Ansible playbooks are defined, take a look at the playbook being run at this [link here π](https://github.com/lewagon/data-engineering-setup/blob/main-automation/automation/vm-ansible-setup/playbooks/setup_vm_part1.yml)
Copy file name to clipboardExpand all lines: _partials/ubuntu_ansible_part2.md
+12-10Lines changed: 12 additions & 10 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -22,15 +22,9 @@ And the playbook should start running! If you're asked if you want VS Code to be
22
22
23
23
β If any errors occur, raise a ticket with a teacher. You can safely run the playbook again.
24
24
25
-
<details>
26
-
<summarymarkdown='span'>β Why two Ansible playbooks?</summary>
27
-
28
-
This second ansible playbook requires GitHub authorisation to fork the `lewagon/data-engineering-challenges` repository and it is also editing some of the Le Wagon recommended **dotfiles**. So we separated the process into two steps.
29
-
</details>
30
-
31
25
### What is the playbook installing?
32
26
33
-
This playbook is installing and configuring a things, while the playbook is running, let's go through them:
27
+
While the playbook is running, let's go through what is being installed and configured.
34
28
35
29
**Python and Poetry**
36
30
@@ -65,22 +59,30 @@ Ubuntu 22.04 has Python pre-installed, but not the version we're going to use. W
65
59
-**gcloud**: Auto-complete for the gcloud CLI tool
66
60
-**ssh-agent**: Saves your SSH password so you only have to enter it once per session.
67
61
-**direnv**: A tool to load `.envrc` files when you `cd` into a directory. Great for loading environment variables.
68
-
- Installing **Spark**: Spark is a distributed data processing framework
62
+
- Installing **Spark**: A distributed data processing framework
69
63
70
64
**Data Engineering Challenges Repository**
71
65
72
66
The challenges that you'll be working on throughout the bootcamp! The playbook is forking the **data-engineering-challenges** repository from **lewagon** to your own GitHub user. Then cloning that repository from your GitHub account down onto your Virtual Machine.
73
67
68
+
π‘ Curious about how Ansible playbooks are defined, take a look at the playbook being run at this [link here π](https://github.com/lewagon/data-engineering-setup/blob/main-automation/automation/vm-ansible-setup/playbooks/setup_vm_part2.yml)
69
+
70
+
<details>
71
+
<summarymarkdown='span'>β Why two Ansible playbooks?</summary>
72
+
73
+
This second ansible playbook requires GitHub authorisation to fork the `lewagon/data-engineering-challenges` repository and it is also editing some of the Le Wagon recommended **dotfiles**. So we separated the process into two steps.
74
+
</details>
75
+
74
76
### Restart Virtual Machine
75
77
76
-
Once the playbook has finished running, you need to completely shutdown your Virtual Machine so that some of the configuration updates (specifically **pyenv** and **Docker**).
78
+
Once the playbook has finished running, you need to completely shutdown your Virtual Machine so that some of the configuration updates properly (specifically **pyenv** and **Docker**).
77
79
78
80
To shutdown your VM, navigate to the GCP Compute Engine Instances [console page π](https://console.cloud.google.com/compute/instances).
79
81
80
82
Select your VM instance and click on the stop button:
81
83
82
84

83
85
84
-
Wait for a few minutes until the VM shows that it is completely off. You may need to refresh the page, the GCP Console doesn't dynamically update.
86
+
Wait for a few minutes until the VM shows that it is completely off. You may need to refresh the page, the GCP Console (web GUI) doesn't dynamically update.
85
87
86
88
When the VM is completely off, turn it on again by selecting the check box next to your instance and clicking **START/RESUME**. Give it a minute to turn on, then connect via VS Code.
Copy file name to clipboardExpand all lines: _partials/vscode_ssh_connection.md
+3-1Lines changed: 3 additions & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -80,8 +80,10 @@ If you want to view your SSH configuration:
80
80
81
81
</details>
82
82
83
-
π¨π¨π¨ **The setup of your local machine is complete. All following terminal commands will be run from within your virtual machine terminal (via VS Code)** π¨π¨π¨
83
+
Congratulations, the setup of you local laptop is complete π We just need to fine tune your Virtual Machine πͺ
84
84
85
85
---
86
86
87
87
# Part 2: Configure your Virtual Machine 1
88
+
89
+
π¨π¨π¨ **All following terminal commands will be run from within your virtual machine terminal, via VS Code** π¨π¨π¨
0 commit comments