Disk partition layout:
# /etc/fstab: static file system information.
#
# Use 'blkid' to print the universally unique identifier for a
# device; this may be used with UUID= as a more robust way to name devices
# that works even if disks are added and removed. See fstab(5).
#
# <file > <system> <mount point> &l;type> <options> <dump> <pass>
# / was on /dev/nvme0n1p1 during installation
UUID=c65d2374-1fba-4bf9-a028-9852c189a71e / ext4 errors=remount-ro 0 1
# /data was on /dev/nvme0n1p4 during installation
UUID=ac4f0e46-b874-4d00-8c90-120ba241f8fd /data ext4 defaults 0 2
# /data1 was on /dev/sdb1 during installation
UUID=8566257a-3500-4ab5-a12f-6768baea74f3 /data1 ext4 defaults 0 2
# /home was on /dev/nvme0n1p3 during installation
UUID=21f313b2-eb50-474e-b86f-fc9f784099c5 /home ext4 defaults 0 2
# swap was on /dev/nvme0n1p2 during installation
UUID=cef07542-2d79-4d02-9eaa-865a39ed6e7c none swap sw 0 0
Filesystem Size Used Avail Use% Mounted on
udev 63G 0 63G 0% /dev
tmpfs 13G 9.4M 13G 1% /run
/dev/nvme0n1p1 59G 1.3G 55G 3% /
tmpfs 63G 0 63G 0% /dev/shm
tmpfs 5.0M 0 5.0M 0% /run/lock
tmpfs 63G 0 63G 0% /sys/fs/cgroup
/dev/nvme0n1p3 234G 61M 222G 1% /home
/dev/nvme0n1p4 564G 73M 536G 1% /data
/dev/sda1 5.5T 89M 5.2T 1% /data1
tmpfs 13G 0 13G 0% /run/user/1000
Prevent password and root login:
- Install sudo (apt-get install sudo)
- Add user to sudo list (usermod -G sudo user)
- in /etc/passwd update root's shell to /usr/sbin/nologin
Prevent remote login:
Kernel version:
4.19.0-17-amd64
Install gcc:
sudo apt-get install gcc
Gcc version:
gcc (Debian 8.3.0-6) 8.3.0
LIBC version:
ldd (Debian GLIBC 2.28-10) 2.28
Install headers:
sudo apt-get install linux-headers-$(uname -r)
Get and install CUDA (downloads ~ 1 GB):
sudo add-apt-repository "deb https://developer.download.nvidia.com/compute/cuda/repos/debian10/x86_64/ /"
sudo apt-get install gnupg
sudo apt-key adv --fetch-keys https://developer.download.nvidia.com/compute/cuda/repos/debian10/x86_64/7fa2af80.pub
sudo add-apt-repository contrib
sudo apt-get update
sudo apt-get -y install cuda
To get nvidia drivers up and running over noveau driver,
sudo shutdown -r now
Adjust paths:
CUDA=/usr/local/cuda-11.4
export PATH=${CUDA}/bin${PATH:+:${PATH}}
export LD_LIBRARY_PATH=${CUDA}/lib64${LD_LIBRARY_PATH:+:${LD_LIBRARY_PATH}}
Check driver:
studen@vangogh:~$ cat /proc/driver/nvidia/version
NVRM version: NVIDIA UNIX x86_64 Kernel Module 470.42.01 Tue Jun 15 21:26:37 UTC 2021
GCC version: gcc version 8.3.0 (Debian 8.3.0-6)
Check compiler:
studen@vangogh:~$ nvcc -V
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2021 NVIDIA Corporation
Built on Wed_Jun__2_19:15:15_PDT_2021
Cuda compilation tools, release 11.4, V11.4.48
Build cuda_11.4.r11.4/compiler.30033411_0
Compile examples in NVIDIA_CUDA-11.4_Samples:
cuda-install-samples-11.4.sh .
cd NVIDIA_CUDA-11.4_Samples
make
Check deviceQuery:
studen@vangogh:~/NVIDIA_CUDA-11.4_Samples$ bin/x86_64/linux/release/deviceQuery
bin/x86_64/linux/release/deviceQuery Starting...
CUDA Device Query (Runtime API) version (CUDART static linking)
Detected 2 CUDA Capable device(s)
Device 0: "NVIDIA RTX A5000"
CUDA Driver Version / Runtime Version 11.4 / 11.4
CUDA Capability Major/Minor version number: 8.6
Total amount of global memory: 24256 MBytes (25434259456 bytes)
(064) Multiprocessors, (128) CUDA Cores/MP: 8192 CUDA Cores
GPU Max Clock rate: 1695 MHz (1.70 GHz)
Memory Clock rate: 8001 Mhz
Memory Bus Width: 384-bit
L2 Cache Size: 6291456 bytes
Maximum Texture Dimension Size (x,y,z) 1D=(131072), 2D=(131072, 65536), 3D=(16384, 16384, 16384)
Maximum Layered 1D Texture Size, (num) layers 1D=(32768), 2048 layers
Maximum Layered 2D Texture Size, (num) layers 2D=(32768, 32768), 2048 layers
Total amount of constant memory: 65536 bytes
Total amount of shared memory per block: 49152 bytes
Total shared memory per multiprocessor: 102400 bytes
Total number of registers available per block: 65536
Warp size: 32
Maximum number of threads per multiprocessor: 1536
Maximum number of threads per block: 1024
Max dimension size of a thread block (x,y,z): (1024, 1024, 64)
Max dimension size of a grid size (x,y,z): (2147483647, 65535, 65535)
Maximum memory pitch: 2147483647 bytes
Texture alignment: 512 bytes
Concurrent copy and kernel execution: Yes with 2 copy engine(s)
Run time limit on kernels: No
Integrated GPU sharing Host Memory: No
Support host page-locked memory mapping: Yes
Alignment requirement for Surfaces: Yes
Device has ECC support: Disabled
Device supports Unified Addressing (UVA): Yes
Device supports Managed Memory: Yes
Device supports Compute Preemption: Yes
Supports Cooperative Kernel Launch: Yes
Supports MultiDevice Co-op Kernel Launch: Yes
Device PCI Domain ID / Bus ID / location ID: 0 / 1 / 0
Compute Mode:
< Default (multiple host threads can use ::cudaSetDevice() with device simultaneously) >
Device 1: "NVIDIA RTX A5000"
CUDA Driver Version / Runtime Version 11.4 / 11.4
CUDA Capability Major/Minor version number: 8.6
Total amount of global memory: 24256 MBytes (25434652672 bytes)
(064) Multiprocessors, (128) CUDA Cores/MP: 8192 CUDA Cores
GPU Max Clock rate: 1695 MHz (1.70 GHz)
Memory Clock rate: 8001 Mhz
Memory Bus Width: 384-bit
L2 Cache Size: 6291456 bytes
Maximum Texture Dimension Size (x,y,z) 1D=(131072), 2D=(131072, 65536), 3D=(16384, 16384, 16384)
Maximum Layered 1D Texture Size, (num) layers 1D=(32768), 2048 layers
Maximum Layered 2D Texture Size, (num) layers 2D=(32768, 32768), 2048 layers
Total amount of constant memory: 65536 bytes
Total amount of shared memory per block: 49152 bytes
Total shared memory per multiprocessor: 102400 bytes
Total number of registers available per block: 65536
Warp size: 32
Maximum number of threads per multiprocessor: 1536
Maximum number of threads per block: 1024
Max dimension size of a thread block (x,y,z): (1024, 1024, 64)
Max dimension size of a grid size (x,y,z): (2147483647, 65535, 65535)
Maximum memory pitch: 2147483647 bytes
Texture alignment: 512 bytes
Concurrent copy and kernel execution: Yes with 2 copy engine(s)
Run time limit on kernels: No
Integrated GPU sharing Host Memory: No
Support host page-locked memory mapping: Yes
Alignment requirement for Surfaces: Yes
Device has ECC support: Disabled
Device supports Unified Addressing (UVA): Yes
Device supports Managed Memory: Yes
Device supports Compute Preemption: Yes
Supports Cooperative Kernel Launch: Yes
Supports MultiDevice Co-op Kernel Launch: Yes
Device PCI Domain ID / Bus ID / location ID: 0 / 33 / 0
Compute Mode:
< Default (multiple host threads can use ::cudaSetDevice() with device simultaneously) >
> Peer access from NVIDIA RTX A5000 (GPU0) -> NVIDIA RTX A5000 (GPU1) : Yes
> Peer access from NVIDIA RTX A5000 (GPU1) -> NVIDIA RTX A5000 (GPU0) : Yes
deviceQuery, CUDA Driver = CUDART, CUDA Driver Version = 11.4, CUDA Runtime Version = 11.4, NumDevs = 2
Result = PASS
Bandwidth test:
studen@vangogh:~/NVIDIA_CUDA-11.4_Samples$ bin/x86_64/linux/release/bandwidthTest
[CUDA Bandwidth Test] - Starting...
Running on...
Device 0: NVIDIA RTX A5000
Quick Mode
Host to Device Bandwidth, 1 Device(s)
PINNED Memory Transfers
Transfer Size (Bytes) Bandwidth(GB/s)
32000000 26.2
Device to Host Bandwidth, 1 Device(s)
PINNED Memory Transfers
Transfer Size (Bytes) Bandwidth(GB/s)
32000000 27.1
Device to Device Bandwidth, 1 Device(s)
PINNED Memory Transfers
Transfer Size (Bytes) Bandwidth(GB/s)
32000000 649.0
Result = PASS
NOTE: The CUDA Samples are not meant for performance measurements. Results may vary when GPU Boost is enabled.
Get CUDA10.0 (not part of core installation, will be dropped):
wget https://developer.nvidia.com/compute/cuda/10.0/Prod/local_installers/cuda-repo-ubuntu1804-10-0-local-10.0.130-410.48_1.0-1_amd64
mv cuda-repo-ubuntu1804-10-0-local-10.0.130-410.48_1.0-1_amd64 cuda-repo-ubuntu1804-10-0-local-10.0.130-410.48_1.0-1_amd64.deb
sudo dpkg -i cuda-repo-ubuntu1804-10-0-local-10.0.130-410.48_1.0-1_amd64.deb
sudo apt-key add /var/cuda-repo-10-0-local-10.0.130-410.48/7fa2af80.pub
sudo apt-get update
sudo apt-get install cuda-libraries-10-0
Use python3.6 (deepMedic stuff, will be dropped):
sudo apt install wget build-essential libreadline-gplv2-dev libncursesw5-dev libssl-dev libsqlite3-dev tk-dev libgdbm-dev libc6-dev libbz2-dev libffi-dev zlib1g-dev libgdbm-compat-dev liblzma-dev
wget https://www.python.org/ftp/python/3.6.14/Python-3.6.14.tgz
tar xvzf Python-3.6.14.tgz
cd Python-3.6.14
./configure --enable-optimizations
make -j 4
Failed or tests not run:
26 test_asyncore skipped
53 test_cmd_line failed
90 test_curses skipped (resource denied)
101 test_devpoll skipped
149 test_gdb skipped
170 test httplib:ConnectionResetError: [Errno 104] Connection reset by peer
173 test_imaplib:ConnectionResetError: [Errno 104] Connection reset by peer
192 test_kqueue skipped
216 test_msilib skipped
219 test_multiprocessing_fork skipped
220 test_multiprocessing_forkserver skipped
221 test_multiprocessing_main_handling skipped
222 test_multiprocessing_spawn skipped
234 test_ossaudiodev skipped (resource denied)
298 test_smtpnet skipped (resource denied)
301 test_socketserver skipped (resource denied)
307 test_startfile skipped
320 test_subprocess skipped
345 test_timeout skipped (resource denied)
346 test_tix skipped (resource denied)
347 test_tk skipped (resource denied)
353 test_ttk_guionly skipped (resource denied)
test_urllib2_localnet: [Errno 104] Connection reset by peer
test_urllib2net skipped (resource denied)
test_urllibnet skipped (resource denied)
test_winconsoleio skipped
test_warnings failed
test_winreg test_winsound skipped (resource denied)
test_winsound skipped (resource denied)
400 test_xmlrpc_net skipped (resource denied)
404 test_zipfile64 skipped (resource denied)
sudo make altinstall