Blog is moving

My blog is moving to http://victormendonca.com/blog/. If you are looking for a specific or older post you are in the right place Otherwise check out my new page for more up to date content.
Showing posts with label Linux. Show all posts
Showing posts with label Linux. Show all posts

Wednesday, October 11, 2017

How to Load GitLab Inside an Iframe

So you are trying to load GitLab inside another page via iframe, and you are not able to. Due to security reasons, this is default behavior for GitLab, and as per the project (see issue 2347, this will not change, and I agree).
However for some internal users this might not be the best approach, so here's how to enable it.
Browse to your install directory and go to your ‘nginx’ folder. If you are not sure where it is, with your GitLab instance running, use the following command (assuming you are using port 8888).
# netstat -anp | grep 8888
tcp        0      0 0.0.0.0:8888                0.0.0.0:*                   LISTEN      19761/nginx
Now use the PID to find where ‘nginx’ is running:
# ps -ef | grep 19761
root     19761  1029  0 13:12 ?        00:00:00 nginx: master process /opt/gitlab/embedded/sbin/nginx -p /var/opt/gitlab/nginx
Browse to the config folder and edit the ‘gitlab-http.conf’
# cd /var/opt/gitlab/nginx/conf

# vim gitlab-http.conf
Add the line proxy_hide_header X-Frame-Options;`` in thelocation /` code block
```http.conf location / { ## Serve static files from defined root folder. ## @gitlab is a named location for the upstream fallback, see below. try_files $uri $uri/index.html $uri.html @gitlab; }
proxy_hide_header X-Frame-Options;

Comment the line `proxy_hide_header X-Frame-Options;` if present

```http.conf
location @gitlab {
    ## If you use HTTPS make sure you disable gzip compression
    ## to be safe against BREACH attack.


    ## https://github.com/gitlabhq/gitlabhq/issues/694
    ## Some requests take more than 30 seconds.
    proxy_read_timeout      300;
    proxy_connect_timeout   300;
    proxy_redirect          off;

    proxy_set_header   X-Forwarded-Proto $scheme;
    proxy_set_header   Host              $http_host;
    proxy_set_header   X-Real-IP         $remote_addr;
    proxy_set_header   X-Forwarded-For   $proxy_add_x_forwarded_for;
    proxy_set_header   X-Frame-Options   SAMEORIGIN;
    #proxy_hide_header X-Frame-Options;

    proxy_pass http://gitlab;
  }
Restart GitLab
# gitlab-ctl restart
ok: run: logrotate: (pid 31748) 1s
ok: run: nginx: (pid 31752) 0s
ok: run: postgresql: (pid 31757) 1s
ok: run: redis: (pid 31766) 0s
ok: run: sidekiq: (pid 31774) 0s
ok: run: unicorn: (pid 31786) 0s
That should be it!!!

Tuesday, September 26, 2017

Bash Special Parameters

Special parameters are set by the shell to store information about aspects of its current state, such as the number of arguments and the exit code of the last command. Special parameters can only be referenced and cannot have it's value assigned.
Specia parameters are: $*, $@, $#, $$, $!, $?, $0, $-, $_
ParameterDefinition
$*List of arguments (as a string)
$@List of arguments (as an array)
$#Number of positional parameters
$$PID of the current shell
$!PID of the last command executed in the background
$?Exit code of the last-executed command
$0Path to the currently running script
$-Current shell option flags
$_Gives the last argument to the previous command

Sunday, September 24, 2017

Atom as Markdown Editor

I have been looking for a good Markdown editor for quite a while, and after a lot of research, I finally I found one.
Some of the key features that were very important to me on a Markdown editor were:
  • Live preview
  • File browser (sidebar)
  • GitHub flavored
  • Fast and non-laggy preview
And not surprisingly, Atom from GitHub was my solution.
Imgur
You might also want to:
  • Change the preferences of the already installed package markdown-preview to GitHub flavored
apm install markdown-preview-auto-open

Saturday, September 23, 2017

Gnome Touchpad Settings Missing in Arch (XPS 13 9360)

I had an issue where the Gnome extension 'Touchpad Indicator' stopped working on my XPS 13 (Arch). After looking a bit further, it seems that the Gnome Touchpad settings had also stopped working. All I could see was the mouse settings, and the touchpad section was completely gone.

Solution:
With Gnome 3.20, xf86-input-synaptics is not longer supported, and you should use xf86-input-libinput instead.
You can check what is installed on your Arch system with pacman -Q | grep input. In my case, I had both packages installed:
$ pacman -Q | grep input
inputproto 2.3.2-1
libinput 1.8.2-1
xf86-input-libinput 0.26.0-1
xf86-input-synaptics 1.9.0-1
xorg-xinput 1.6.2-1
Remove xf86-input-synaptics and any configuration file (like /etc/X11/xorg.conf.d/50-synaptics.conf), install xf86-input-libinput and reboot. That should get your configuration working again.

Wednesday, September 20, 2017

Additional Status for VIM with Powerline

Would you like to have more information displayed while reading files in VIM? Powerline is a great utility for that.
Imgur
In it's default config, it displays:
  • Current mode (normal, insert, visual)
  • Git branch
  • File name
  • File encoding
  • Script type
  • File view percentage
  • Line number
To install it on Arch:
sudo pacman -Ss python-powerline powerline powerline-vim
Add the line below to your ~/.vimrc
set laststatus=2

Note: If you get the error below
Traceback (most recent call last):
  File "", line 9, in 
ImportError: No module named powerline.vim
An error occurred while importing powerline module.
This could be caused by invalid sys.path setting,
or by an incompatible Python version (powerline requires
Python 2.6, 2.7 or 3.2 and later to work). Please consult
the troubleshooting section in the documentation for
possible solutions.
If powerline on your system is installed for python 3 only you
should set g:powerline_pycmd to "py3" to make it load correctly.
Unable to import powerline, is it installed?
Press ENTER or type command to continue
Modify either ~/.vimrc (or /etc/vimrc if you want the fix available for multiple users) by adding the line below:
let g:powerline_pycmd = 'py3'

Tuesday, September 19, 2017

Bash: Parameter Expansion (Substitution)

Default Usage

${PARAMETER}
Expand parameter value. It can be used for:
  • Separating characters from variable names
    • cd /home/${user}
  • Positional parameter higher than 9
    • echo "Argument 10 is: ${10}"

Indirection

${!PARAMETER}
Expands to the value of the variable named by the value of parameter.
var=var1
var1=temp

$ echo ${!var}
temp

Case Modification

${PARAMETER^}, ${PARAMETER^^} ${PARAMETER,}, ${PARAMETER,,} ${PARAMETER~}, ${PARAMETER~~}
OperatorAction
^Changes first character to upper case
^^Changes all characters to upper case
,Changes first character to lower case
,,Changes all characters to lower case
~Inverts case of first character
~~Inverts case of all characters

Variable Name (Prefix) Expansion

${!PREFIX*} ${!PREFIX@}
Expands to a list of all set variable names beginning with the string in PREFIX.
$ echo ${!XDG*}
XDG_CURRENT_DESKTOP XDG_MENU_PREFIX XDG_RUNTIME_DIR XDG_SEAT XDG_SESSION_DESKTOP XDG_SESSION_ID XDG_SESSION_TYPE XDG_VTNR

Substring Removal

${PARAMETER#PATTERN} ${PARAMETER##PATTERN} ${PARAMETER%PATTERN} ${PARAMETER%%PATTERN}
With #, it removes the mathing pattern from the beggining of the variable, where # removes the shortest match, and ## removes the longest.
Note the empty spaces in the example below.
substr="the quick brown fox jumps over the lazy dog"

$ echo ${substr#* }
quick brown fox jumps over the lazy dog

$ echo ${substr##* }
dog
Removing path from a file:
mlog=/var/log/clamav/freshclam.log

$ echo ${mlog##*/}
freshclam.log
The operator % does the same, but at the end of the file.
$ echo ${substr% *}
the quick brown fox jumps over the lazy

$ echo ${substr%% *}
the
Changing the extension of a file
file=123.txt

$ echo ${file%.*}
123

$ echo ${file%.*}.log
123.log

Search and Replace

${PARAMETER/PATTERN/STRING} ${PARAMETER//PATTERN/STRING} ${PARAMETER/PATTERN} ${PARAMETER//PATTERN}
The main diffence is that a single / substitutes the first occurrence, while double // substitute all occurences:
$ echo $substr 
the quick brown fox jumps over the lazy dog

$ echo ${substr/the/da}
da quick brown fox jumps over the lazy dog

$ echo ${substr//the/da}
da quick brown fox jumps over da lazy dog

$ echo ${substr/the}
quick brown fox jumps over the lazy dog

$ echo ${substr//the}
quick brown fox jumps over lazy dog

Anchoring

You can use # and % to anchor to the beginning and end respectively
var1=00000000

$ echo ${var1/#0/1}
10000000

$ echo ${var1/%0/1}
00000001

Offset and Lenght

${PARAMETER:OFFSET} ${PARAMETER:OFFSET:LENGTH}
  • Offset removes the amount of characters as specified
  • Lenght prints the specified character lenght after offset
Note: You can also use a negative value for offset and lenght, which will be calculated from the end of the file
$ echo ${substr}
the quick brown fox jumps over the lazy dog
123456789...
Removes first 9 chars
$ echo ${substr:9}
brown fox jumps over the lazy dog
Removes first 3 chars and print the next 5
$ echo ${substr:4:5}
quick

Use Default

${PARAMETER:-WORD} ${PARAMETER-WORD}
If parameter unset (or null if using :), expand to word.
$ my_var=gru
$ echo ${my_var:-yyz}
gru

$ unset my_var
$ echo ${my_var:-yyz}
yyz

Use Alternate Value

${PARAMETER:+WORD} ${PARAMETER+WORD}
If parameter is set (or null if using :), expand to word.
$ my_var=gru
$ echo ${my_var:+yyz}
yyz

$ unset my_var
$ echo ${my_var:+yyz}

Use Default And Assign

${PARAMETER:=WORD} ${PARAMETER=WORD}
If parameter unset (or null if using :), expand to word and assign parameter to the value of word.
my_var=gru

$ echo ${my_var:=yyz}
gru

$ unset my_var
$ echo ${my_var:=yyz}
yyz

$ echo $my_var
yyz

Display Error

${PARAMETER:?WORD} ${PARAMETER?WORD}
If parameter unset (or null if using :), display error with word as appendix, otherwise expand parameter.
my_var=gru

$ echo ${my_var:?Not set}
gru

$ unset my_var
$ echo ${my_var:?Not set}
bash: my_var: Not set

Reference:

Wednesday, August 30, 2017

Alternatives to top

Looking for alternatives to your usual top command? Here are some options.

htop

htop is based on ncurses and is compatible with most Linux (and Unix) systems. It's also in most the official repos for most distros.
Features:
- Mouse clicks (due to ncurses)
- Defaults to multi-CPU view
- Memory shown in GB
htop

vtop

vtop takes more of a graphical approach to top, while concentrating more on simplicity. It displays CPU and Memory usage live charts, as well as a running process list. It runs on Node.js and can be easily installed with npm install -g vtop.
Features:
- Mouse clicks
- Themes - Live chart (CPU, memory)
- Process list
vtop

gtop

gtop takes the graphical interface of vtop to another level. Just like vtop, it also displays a live chart of both CPU and memory, however gtop adds a network live chart to the list, and pie charts for both memory and swap, as well as storage usage.
gtop also runs on Node.js, and can easily be installed with npm install gtop -g.
Features:
- Live chart (CPU, memory/swap, network)
- Pie chart (memory, swap, disk usage)
- Process list
gtop

s-tui

While s-tui is not a process monitoring utility, it displays great information about your process status. It's python based and uses little resource. You can easily install it with sudo pip install s-tui.
Features:
- Processor info
- Live charts (CPU freq, utilization, temperature and power usage)
- Built-in CPU stress testing
Website: https://amanusk.github.io/s-tui/

Wednesday, January 11, 2017

Bash - Command Substitution, Subshell and Codeblock

Quick and simple differences between using $(..)(..) and {..;}.
  • $(command) or `command` - Command substitution (the output of a command replaces the command name) and is executed in a subshell. Please give preference to $(..) for better readability
  • ( command ) - Command list is executed in a subshell
  • { command ;} - Command list is executed in the current shell
Executing code
$ ( echo 123 )
123

$ $(echo 123) # this executes the output of the command
123: command not found

$ { echo 123 ;}
123
Fiding subshell level with built-in BASH_SUBSHELL variable
$ ( echo $BASH_SUBSHELL )
1
$ ( ( echo $BASH_SUBSHELL ) )
2

$ $($BASH_SUBSHELL)
1: command not found

$ { echo $BASH_SUBSHELL ;}
0
Passing variables to subshell
$ var1=1

$ ( echo $var1 )
1

$ $(echo $var1)
1: command not found

$ { echo $var1 ;}
1
Changing and getting variables values in subshell
$ var=1

$ ( var=2 ) ; echo $var
1

$ $(var=2) ; echo $var
1

# Because we did not create a subshell, 
# variable is available to original shell
$ { var=2 ;} ; echo $var 
2

Other usages

Example 1:
Multiple commands and multiple lines. This can be used with (..) and {..}.
$ { echo 1 ; echo 2 ; echo 3; }
1
2
3

$ { 
> echo 1
> echo 2
> echo 3;
> }
1
2
3
Example 2:
Changing the environment for subshell with separate environment. Here, set -u is only set in the subshell, so only the first echo will error out.
unset var1 var2
(
  set -u
  echo $var2
) 
echo $var1

Friday, December 18, 2015

Bash Scripting - Best Practices

1 - Readability

1.1 - Indentation

There are 3 commonly used indentation practices for Bash (I prefer the first method, however all 3 are "accepted"):
  • 2 spaces
  • 4 spaces
  • tabs (usually 8 spaces)
All examples will be shown using the first indentation method, however for reference here's a comparison between all 3.
Example:
## 2 spaces
if ...
  command
else ...
  command
fi

## 4 spaces
if ...
    command
else ...
    command
fi

## Tabs
if ...
        command
else ...
        command
fi
Indentation for if conditional statements
if [ test ] ; then
  command
elif [ test ] ; then
  command
else
  command
fi
Indentation for for statements
for a b c in $A ; do
  command
done
Indentation for while and until loops
while [ true ] ; do
  command
done
Indentation for case statements
case $VAR in
  true) command ;;
  false)
    command1
    command2
    ;;
  *)
    if [ test ] ; then
      command
    fi
    ;;
esac
Indentation for functions
_funct_do_var() {
  commands
}

1.2 - Comments

Code Comments

You should always comment your code. This will make it easier for others to understand, as well as for yourself should you need to change the script a few months (or years) later.
Make sure that the comments make sense. Do not try to save on typing as additional comments can save you (or someone else) a lot of time in the future.
Example: Commonly used comments
startTMATE() {
  ## Starts tmate handling
  # Launch tmate in a detached state
  $TMATE_BIN -S $TMT_SOCKET new-session -d

  # Blocks until the SSH connection is established
  $TMATE_BIN -S $TMT_SOCKET wait tmate-ready

  # Prints the SSH connection string
  $TMATE_BIN -S $TMT_SOCKET display -p '#{tmate_ssh}' > $LOG

  # Prints the read-only SSH connection string
  $TMATE_BIN -S $TMT_SOCKET display -p '#{tmate_ssh_ro}' >> $LOG
}
Example: In-line comments
## Sets up package manager aliases based on distro
# Settings for Ubuntu
if [[ "$DISTRO" =~ (Ubuntu|LinuxMint) ]] ; then
  alias aptdate='sudo apt-get update'         # Updates package list
  alias aptgrade='sudo apt-get upgrade'       # Updates all packages
  alias apts='apt-cache search'               # Search for package
  alias aptrm='sudo apt-get remove'           # Removes package

  aptinst() {                                # Install package
    if [ -d "${HOME}/bin/var/log" ] ; then
      LOG=${HOME}/bin/var/log/$(hostname)-package-install.log
    fi

    echo -e "$(date)\t-\tInstalling packages: $*" >> "$LOG"
    sudo apt-get install "$*"
  }
fi

Block Comment

Here Document
You can use a here document (EOF) to "trick" Bash in creating block comments.
Example:
alt text
You can read a bit on my other post here.
Bash Built-in Command
You can also use Bash's bultin :, which does nothing.
Here's the definition from Bash's man page:
: [arguments]
    No effect; the command does nothing beyond expanding arguments and
    performing any specified redirections. A zero exit code is returned.
Example:
: '
 your comments here
'

1.3 - Lines

Breaking Line of Code
When breaking long lines of code with \, indent the new lines.
Example:
[ -f "/etc/cups/cupsd.conf" ] && echo \
  "The CUPS configuration file exists" 
Using Empty Lines
You can use empty lines to keep your code clean, even inside code blocks.
Example:
# Downloads this weeks photos
if [ "$THIS_WEEKS_PHOTOS_URL" ] ; then
  cd ${UNSPLASH_DIR}
  for PHOTO_URL in $THIS_WEEKS_PHOTOS_URL ; do

    # Checks if jpg is in the URL
    if [[ $(echo $PHOTO_URL | grep -qi jpg ; echo $?) -eq 0 ]] ; then
      PHOTO_FILE_NAME=$(echo $PHOTO_URL | awk -F/ '{print $4}')

    # else, let's add it
    else
      PHOTO_FILE_NAME=$(echo $PHOTO_URL | awk -F/ '{print $4}').jpg
    fi

    # Get the photo and save as file name
    wget --progress=bar $PHOTO_URL -O $PHOTO_FILE_NAME
  done
fi

1.4 - Misc

  • Avoid the use of back-ticks ` for command substitution. Use $(...) for better readability

2 - Headers and Sections

2.1 - Script Header

Add descriptive headers to the script outlining it's name, what it does, usage and possibly version.
#!/bin/bash

################################################################################
################################################################################
# Name:          tmate.sh
# Usage:
# Description:   Runs tmate and outputs remote string to file
# Created:       2014-10-31
# Last Modified:
# Copyright 2014, Victor Mendonca - http://wazem.org
# License: Released under the terms of the GNU GPL license
################################################################################
################################################################################

2.2 - Sections

Divide your script into sections.
Put all global script variables in one section
#-------------------------------------------------------------------------------
# Sets variables
#-------------------------------------------------------------------------------
Put all functions in one section
#-------------------------------------------------------------------------------
# Functions
#-------------------------------------------------------------------------------
Put all major work in one section (where functions get called)
#-------------------------------------------------------------------------------
# Starts script
#-------------------------------------------------------------------------------

3 - Variables

3.1 - Naming

  • Give meaningful names to variables
  • When using uppercase, make sure the variable is not already being used
  • Avoid starting variables with _

3.2 - Calling

Double quote variables to prevent globbing or word splitting.

4 - Functions

Try to define your functions at the beginning of the script (see sections).

4.1 - Naming

  • Should start with lower case
  • It's good to start with _
  • Upper and lower case are also good
Example:
_lowerIt () {
  BASH_MAIN_VERSION=$(echo $BASH_VERSION | awk -F. '{print $1}')
  if [[ $BASH_MAIN_VERSION -lt 4 ]] ; then
    echo "Not supported with your version of bash"
    return
  fi

  if [ "$1" ] ; then
    echo ${1,,}
  fi
}

4.2 - Local Variables

Keep your function variables local if they are not being used outside of the function.
Example: This script will output "$2400" only once
_myFunction() {
  local extract
  extract='$2400'
  echo "$extract"
}

_myFunction
echo "$extract"

5- Code Smart

New and Deprecated Features
Get to know your version of Bash and any features it may have.
Example: Old and new arithmetic expansion
# New way for arithmetic expansion
$ echo $((4*25/2))
50

# Old way for arithmetic expansion, which will be deprecated
$ echo $[4*25/2]
50
New features
Bash 4 packs new features (like parameter expansion and output redirects). Get familiar with them as they can save you a lot of time.
Examples: Parameter expansion
$ title="bash best practices"

## Case substitution
$ echo ${title^^}
BASH BEST PRACTICES

## String substitution
$ echo ${title/best/worst}
bash worst practices
Examples: stdout and stderr output redirect
# New
ls 123 &> /dev/null

# Old
ls 123 > /dev/null 2>&1

6 - Software

You can also use software to help with your scripts.

6.1 - Syntax Highlighting

Most of today's editors include syntax highlighting, even VIM. Syntax highlighting will help you catch mistakes like a unclosed bracket or quote.

6.2 - Lint

Some editors, like Sublime Text include plugins for lint software. They can be of great help for you coding.

6.3 - Online Tools

There are also sites that can help checking your script (similar to lint), or a small set of commands.
  • explainshell.com - match command-line arguments to their help text
  • shellcheck.net - automatically detects problems in sh/bash scripts and commands

References