GitLab搭建及踩坑
GitLab搭建及踩坑

GitLab搭建及踩坑

服务器环境:

OS: Ubuntu18.04-LTS
CPU: 2核
内存: 4G

Docker搭建GitLab

  • 拉取gitlab-ce镜像
root@iZ2ze26pixxe9t9kmg6tvhZ:# docker pull gitlab/gitlab-ce
  • 创建gitlab容器的数据映射目录

其中config目录映射gitlab容器的配置文件目录,
data目录映射gitlab容器的数据目录,
logs目录映射gitlab容器的日志目录

root@iZ2ze26pixxe9t9kmg6tvhZ:/home# mkdir gitlab
root@iZ2ze26pixxe9t9kmg6tvhZ:/home# cd gitlab
root@iZ2ze26pixxe9t9kmg6tvhZ:/home/gitlab# mkdir data config logs
root@iZ2ze26pixxe9t9kmg6tvhZ:/home/gitlab# ls
config  data  logs
root@iZ2ze26pixxe9t9kmg6tvhZ:/home/gitlab# 
  • 创建gitlab容器

gitlab容器一共暴露了三个端口: 22/80/443
由于未使用SSL证书, 443端口可以不映射
22端口用于ssh传输, 也可以不映射
只使用容器暴露的80端口, 映射到宿主机的83端口

root@iZ2ze26pixxe9t9kmg6tvhZ:/home/gitlab# docker run -d -p 83:80 --name mygitlab -- restart always -v /home/gitlab/config/:/etc/gitlab -v /home/gitlab/data/:/var/opt/gitlab -v /home/gitlab/logs/:/var/log/gitlab gitlab/gitlab-ce:latest
  • 修改gitlab的配置文件

gitlab的配置文件为容器内的/etc/gitlab/gitlab.rb
由于映射到了容器外的/home/gitlab/config目录下, 可以直接在容器外修改:

root@iZ2ze26pixxe9t9kmg6tvhZ:/home/gitlab/config# vim gitlab.rb

gitlab.rb配置文件默认是全注释的.
修改配置文件中对外暴露的地址:

## GitLab URL
##! URL on which GitLab will be reachable.
##! For more details on configuring external_url see:
##! https://docs.gitlab.com/omnibus/settings/configuration.html
#configuring-the-external-url-for-gitlab
##!
##! Note: During installation/upgrades, the value of the environment variable
##! EXTERNAL_URL will be used to populate/replace this value.
##! On AWS EC2 instances, we also attempt to fetch the public hostname/IP
##! address from AWS. For more details, see:
##! https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/instancedata-data-retrieval.html
external_url 'http://gitlab.zack.net.cn'
  • 修改宿主机的nginx配置文件, 添加端口转发规则

新增一条端口转发规则, 将主机名为gitlab.zack.net.cn的80端口请求转发到gitlab容器映射的83端口,
然后重载nginx配置文件

server{
                listen          80;
                server_name     gitlab.zack.net.cn;
                location /{
                        # HOST配置以及域名传递
                        proxy_set_header Host $host;
                        proxy_set_header X-Real-IP $remote_addr;
                        proxy_set_header REMOTE-HOST $remote_addr;
                        # 不使用服务端端口响应
                       proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
                        # 转发到83端口
                        proxy_pass http://127.0.0.1:83/;
                }
        }
  • 重启gitlab容器进行访问
root@iZ2ze26pixxe9t9kmg6tvhZ:/home/gitlab# docker restart mygitlab

等待重启后访问http://gitlab.zack.net.cn, 设置root账户密码,然后登陆gitlab

Gitlab踩坑

服务器内存不足导致的卡顿问题

gitlab一般要求4核4G内存的起步配置, 对内存消耗较大, 这也是功能强大的原因.由于服务器为2核4G, 同时运行的还有其他服务, 导致gitlab运行期间服务器卡顿, 每次pull/push操作都会失去响应几分钟, CPU占用100%, 内存占满, ssh都会被强行断开连接.

解决办法:设置一个较大的内存交换区, 在内存紧张时将不活跃的内存区域释放出来, 存到交换区
  • 查看系统swap区

这里是已经设置过的swap区, 大小为4G, 云服务器swap区默认没有或者较小

root@iZ2ze26pixxe9t9kmg6tvhZ:~# sudo swapon -s 
Filename              Type     Size      Used   Priority
/mnt/swap             file     4194304   84224  -2
root@iZ2ze26pixxe9t9kmg6tvhZ:~# 
  • 禁用swap区

如果存在swap区, 可以先禁用
如果swap没有, 无需禁用,直接创建即可

root@iZ2ze26pixxe9t9kmg6tvhZ:~# sudo swapoff /swapfile
  • 创建swap区文件

swap区大小为 bs * count
一般情况下为/mnt/swap, 也有可能为/data/swap

root@iZ2ze26pixxe9t9kmg6tvhZ:~# dd if=/dev/zero of=/mnt/swap bs=512 count=8388616
  • 将创建的swap分区文件做成swap分区
root@iZ2ze26pixxe9t9kmg6tvhZ:~# mkswap /mnt/swap
  • 查看内核参数vm.swappiness
root@iZ2ze26pixxe9t9kmg6tvhZ:~# cat /proc/sys/vm/swappiness

如果为0, 根据实际需要设置为60

root@iZ2ze26pixxe9t9kmg6tvhZ:~# sysctl -w vm.swappiness=60
  • 启用swap分区
root@iZ2ze26pixxe9t9kmg6tvhZ:~# swapon /mnt/swap
  • 再次检查swap分区情况
root@iZ2ze26pixxe9t9kmg6tvhZ:~# sudo swapon -s 
Filename    Type    Size     Used   Priority
/mnt/swap   file    4194304  84224  -2
root@iZ2ze26pixxe9t9kmg6tvhZ:~# 
内存消耗过大导致的卡顿问题

随着gitlab服务启动的时间越来越长, 其内存占用越来越大, 内存也越来越紧张

  1. 未启动gitlab时
root@iZ2ze26pixxe9t9kmg6tvhZ:~# free -h
              total        used        free      shared  buff/cache   available
Mem:           3.9G        958M        2.3G        304K        621M        2.7G
Swap:          4.0G         98M        3.9G
root@iZ2ze26pixxe9t9kmg6tvhZ:~# 
  1. 刚启动时, 内存还够用
root@iZ2ze26pixxe9t9kmg6tvhZ:~# free -h
              total        used        free      shared  buff/cache   available
Mem:           3.9G        2.3G        182M         27M        1.3G        1.2G
Swap:          4.0G         98M        3.9G
root@iZ2ze26pixxe9t9kmg6tvhZ:~# 
  1. 使用一段时间后, 内存开始紧张
root@iZ2ze26pixxe9t9kmg6tvhZ:~# free -h
              total        used        free      shared  buff/cache   available
Mem:           3.9G        3.1G        171M         40M        554M        552MSwap:          4.0G        101M        3.9G
root@iZ2ze26pixxe9t9kmg6tvhZ:~# 
  1. 查看系统资源使用情况, 发现gitlab占用了超过80%的内存
root@iZ2ze26pixxe9t9kmg6tvhZ:~# ps aux|head -1;ps aux|grep -v PID|sort -rn -k +4|head
USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
root     18176  0.1 15.2 3544740 616416 ?      Sl   Apr15   2:06 java -Djava.security.egd=file:/dev/./urandom -jar -Duser.timezone=GMT+08 /app.jar
998       5460  0.5 15.1 1096440 611368 ?      Sl   15:31   0:02 unicorn worker[3] -D -E production -c /var/opt/gitlab/gitlab-rails/etc/unicorn.rb /opt/gitlab/embedded/service/gitlab-rails/config.ru
998       4774  0.2 14.4 1036824 584224 ?      Sl   15:27   0:01 unicorn worker[1] -D -E production -c /var/opt/gitlab/gitlab-rails/etc/unicorn.rb /opt/gitlab/embedded/service/gitlab-rails/config.ru
998       5851  0.4 14.0 1036824 568900 ?      Sl   15:34   0:00 unicorn worker[2] -D -E production -c /var/opt/gitlab/gitlab-rails/etc/unicorn.rb /opt/gitlab/embedded/service/gitlab-rails/config.ru
998      30297  0.0 13.9 1029252 562264 ?      Sl   Apr15   0:49 unicorn master -D -E production -c /var/opt/gitlab/gitlab-rails/etc/unicorn.rb /opt/gitlab/embedded/service/gitlab-rails/config.ru
998       6200  0.3 13.8 1034660 559924 ?      Sl   15:37   0:00 unicorn worker[0] -D -E production -c /var/opt/gitlab/gitlab-rails/etc/unicorn.rb /opt/gitlab/embedded/service/gitlab-rails/config.ru
998      30153  0.7 13.6 1177528 552084 ?      Ssl  Apr15   9:33 sidekiq 5.2.7 gitlab-rails [0 of 8 busy]
999       1333  0.0  4.8 1598884 195808 ?      Ssl  Apr15   1:05 mysqld
992      30150  0.5  4.6 689920 187392 ?       Ssl  Apr15   6:23 /opt/gitlab/embedded/bin/prometheus --web.listen-address=localhost:9090 --storage.tsdb.path=/var/opt/gitlab/prometheus/data --config.file=/var/opt/gitlab/prometheus/prometheus.yml
998      30316  0.1  1.9 1342916 77756 ?       Sl   Apr15   1:35 ruby /opt/gitlab/embedded/service/gitaly-ruby/bin/gitaly-ruby 398 /var/opt/gitlab/gitaly/internal_sockets/ruby.1
root@iZ2ze26pixxe9t9kmg6tvhZ:~# 
解决办法: 限制gitlab对内存的使用
  • 修改映射到宿主机的gitlab配置文件
root@iZ2ze26pixxe9t9kmg6tvhZ:~# vim /home/gitlab/config/gitlab.rb
# 修改可以使用的缓存大小200MB-300MB
unicorn['worker_memory_limit_min'] = "200 * 1 << 20"
unicorn['worker_memory_limit_max'] = "300 * 1 << 20"
# 减少sidekiq并发数
sidekiq['concurrency'] = 6
# 修改数据库缓存大小
postgresql['shared_buffers'] = "128MB"
# 修改数据库并发数
postgresql['max_worker_processes'] = 6
# 超时时间
unicorn['worker_timeout'] = 60
# 修改为和cpu核心数一致
unicorn['work_processes'] = 2