Best Ways to Edit Files over SSH (Linux): sed vs Perl and All the Better Options

Summary (TL;DR)

Simple, linebased tweaks sed -i.bak
Advanced regex / multiline / logic perl -pi -e
Structured configs jq (JSON), yq (YAML), crudini (INI), xmlstarlet (XML)
Bulk & repeatable changes version control + scripts or Ansible
Always dryrun, back up, and scope edits

---

Table of Contents

1. Safety Rules (Do This Every Time)

2. sed vs Perl: When to Choose Which

3. Essential Alternatives (Beyond sed/Perl)

4. The Cookbook: Common Editing Tasks

5. DryRun Patterns & Safe Rollback

6. Structured Config Editing (JSON/YAML/INI/XML)

7. Bulk, CrossFile & CrossServer Operations

8. Tips for Editors (vim/nvim, nano, ed/ex)

9. Troubleshooting & Pitfalls

10. Final Checklist (Print & Stick on the Wall)

---

1) Safety Rules (Do This Every Time)

• Preview first (no inplace changes):

  sed 's/OLD/NEW/g' file | diff -u file -
  perl -pe 's/OLD/NEW/g' file | diff -u file -
  

• Create a backup you can later clean:

  sed -i.bak 's/OLD/NEW/g' file
  perl -pi.bak -e 's/OLD/NEW/g' file
  

• Scope precisely to only the intended files:

  git ls-files '*.conf' | xargs -r sed -i.bak 's/OLD/NEW/g'
  find config -type f -name '*.ini' -print0 | xargs -0 -r perl -pi.bak -e 's/OLD/NEW/g'
  

• Fail fast in scripts:

  set -euo pipefail
  

• Privileged files: write safely via sudo tee to avoid partial writes:

  sed 's/OLD/NEW/' file | sudo tee file >/dev/null
  

• Atomic updates for critical configs: write to temp mv into place.

---

2) sed vs Perl: When to Choose Which

Scenario	sed	Perl
Oneliners, simple substitutions, delete/comment lines	Tiny & fast	Overkill but fine
Extended regex (groups, alternation)	with -E	Builtin
Lookaround, lazy quantifiers, named groups	Limited	Full PCRE
Multiline edits	Possible but awkward	-0777, /s
Consistent -i behavior across distros	BSD vs GNU quirks	Stable -pi
Very large files	Very fast	Fast enough

Rules of thumb:

• Pick sed for common, linebased edits and portability on Linux servers.

• Pick Perl when your regex needs teeth (lookarounds, multiline stanzas, conditional logic).

---

3) Essential Alternatives (Beyond sed/Perl)

Use the right tool for the data. Text is not always "just text".

• awk - Great for columnar transforms, selecting fields, and conditional edits.

• vim/nvim - Exmode & macros for complex, repeatable edits across buffers/files.

• nano - Quick manual fix in a pinch; easy for nonvim users.

• ed/ex - Scriptable line editors; perfect for automation in minimal environments.

• ripgrep (rg) + sd - Ultrafast search + ergonomic replace (sd is a friendlier sed for many cases).

• jq / yq / crudini / xmlstarlet - Edit JSON/YAML/INI/XML structurally (no fragile regex).

• Augeas (augtool) - Policydriven config editing using lenses (safer for many system configs).

• Templating - Generate configs instead of patching them: envsubst, gomplate, jinja2-cli.

• Orchestration - Idempotent, documented edits at scale: Ansible (lineinfile, blockinfile, template).

• moreutils's sponge - Avoid truncation when reading & writing the same file: cmd | sponge file.

---

4) The Cookbook: Common Editing Tasks

4.1 Global Replace (with safe delimiter)

sed -i.bak 's|/var/www/app|/srv/app|g' /etc/app.conf
perl -pi.bak -e 's|/var/www/app|/srv/app|g' /etc/app.conf

4.2 Append a Line After a Match

sed -i.bak '/^\[server\]/a max_connections = 200' /etc/app.ini
perl -pi.bak -e 's/^\[server\]\n/[server]\nmax_connections = 200\n/' /etc/app.ini

4.3 Delete Lines Matching a Pattern

sed -i.bak '/^#\s*DEBUG/d' /etc/app.conf
perl -ni.bak -e 'print unless /^#\s*DEBUG/' /etc/app.conf

4.4 Comment / Uncomment a Directive

# Comment
sed -i.bak 's/^(Listen 8080)/# \1/' /etc/app.conf
perl -pi.bak -e 's/^(\s*Listen 8080)/# $1/' /etc/app.conf

# Uncomment
sed -i.bak 's/^#\s*(Listen 8080)/\1/' /etc/app.conf
perl -pi.bak -e 's/^#\s*(Listen 8080)/$1/' /etc/app.conf

4.5 Change a Key=Value Safely (Idempotent)

sed -i.bak -E 's|^(\s*timeout\s*=\s*).*|\1"60s"|' /etc/app.conf
perl -pi.bak -e 's/^(\s*timeout\s*=\s*).*/${1}"60s"/' /etc/app.conf

4.6 Within a Section Only (Bounded Range)

# sed range between headings
sed -i.bak '/^\[server\]/,/^\[/{s/^port=.*/port=9090/}' /etc/app.ini

# Perl stateful section tracking
perl -pi.bak -e '$in=1 if /^\[server\]/; $in=0 if /^\[.*\]/ && !/^\[server\]/; $in && s/^port=.*/port=9090/' /etc/app.ini

4.7 MultiLine Insertion (Prefer Perl)

perl -0777 -pe 's/(\[logging\]\n)/$1level = "info"\nformat = "json"\n/s' -i.bak /etc/app.ini

4.8 Version Bump (Minor)

perl -pi.bak -e 's/^version\s*=\s*(\d+)\.(\d+)/"version=".( $1 ).".".($2+1)/e' app.ini

4.9 CRLF Cleanup (Before Regex Edits)

sed -i.bak 's/\r$//' file
perl -pi.bak -e 's/\r$//' file

4.10 Column Edits with awk

# Increase 3rd column by 10
awk '{ $3 += 10 }1' input.tsv > out.tsv

---

5) DryRun Patterns & Safe Rollback

• Preview changes with diff:

  perl -pe 's/foo/bar/g' file | diff -u file - | sed -n '1,200p'
  

• Gitaware bulk edits:

  git ls-files -z '*.conf' | xargs -0 -r perl -pi.bak -e 's/OLD/NEW/g'
  git diff --name-only | xargs -r -n1 sed -n '1,50p'
  

• Rollback quickly:

  git restore --source=HEAD -- :/
  # or restore a single file from its backup
  cp file.bak file
  

• Cleanup backups after validation:

  find . -type f -name '*.bak' -delete
  

---

6) Structured Config Editing (JSON/YAML/INI/XML)

Regex isn't a parser. Prefer structureaware tools.

• JSON jq

  jq '.logging.level="info"' config.json | sponge config.json
  

• YAML yq

  yq -i '.server.port = 9090' config.yaml
  

• INI crudini

  crudini --set /etc/app.ini server port 9090
  

• XML xmlstarlet

  xmlstarlet ed -u '/config/server/port' -v 9090 config.xml > config.xml.tmp && mv config.xml.tmp config.xml
  

• Augeas (augtool) - Lensbased, safe edits for many system files:

  augtool set /files/etc/app.ini/server/port 9090
  augtool save
  

---

7) Bulk, CrossFile & CrossServer Operations

• Find only the targets (speed + safety):

  rg -l --hidden --glob '!vendor/*' 'pattern' | xargs -r perl -pi.bak -e 's/OLD/NEW/g'
  

• Parallelize for speed (carefully):

  rg -l 'OLD' | parallel -j4 perl -pi.bak -e 's/OLD/NEW/g' {}
  

• Idempotent at scale: Ansible
  Use modules: lineinfile, blockinfile, replace, template. Example:

  - name: Ensure timeout is 60s
   ansible.builtin.lineinfile:
   path: /etc/app.conf
   regexp: '^\s*timeout\s*='
   line: 'timeout = "60s"'
   backup: yes
  

• Version everything with Git or Etckeeper for /etc.

---

8) Tips for Editors (vim/nvim, nano, ed/ex)

• vim Exmode oneshot:

  vim -Es +'g/^#\s*DEBUG/d' +wq /etc/app.conf
  

• Across many files:

  vim -p $(git ls-files '*.conf') \
   +'argdo %s/OLD/NEW/g | update' +qa
  

• Macros & visual block: Perfect for column edits or repeated transformations.

• nano: Easy, minimal; enable line numbers and soft wrap in ~/.nanorc.

• ed/ex: Deterministic, scriptable in minimal shells/containers.

---

9) Troubleshooting & Pitfalls

• Nothing changed Your regex didn't match. Verify:

  rg -n 'your-regex' file
  

• Overmatching: Anchor with ^/$, or use word boundaries (\b in Perl).

• Delimiter hell: Switch delimiters: s|/a/b|/x/y|g.

• BSD vs GNU sed -i: On macOS, -i'' means "no backup"; on Linux GNU sed, -i.bak is standard.

• Locales: For bytewise speed & predictability: LC_ALL=C.

• Binary or huge files: Don't sed/perl binaries. Stream or split large files if memory is tight.

---

10) Final Checklist (Print & Stick on the Wall)

• Preview with diff

• Back up (-i.bak)

• Scope the target set (git ls-files/find/rg)

• Choose the right tool (sed/Perl/jq/yq/crudini/xmlstarlet)

• Apply atomically for critical configs

• Commit to version control & document the change

---

CopyReady BulkEdit Template (Safe & GitAware)

#!/usr/bin/env bash
set -euo pipefail

PATTERN='^(\s*timeout\s*=\s*).*$'
REPLACEMENT='${1}"60s"'

# Scope: only tracked INI files under config/
git ls-files -z 'config/**/*.ini' \
| xargs -0 -r perl -pi.bak -e "s/${PATTERN}/${REPLACEMENT}/"

echo "Changes staged:"; git status --porcelain

echo "Preview diff (first 200 lines):"
git diff | sed -n '1,200p'

# Rollback (if needed): git restore --source=HEAD -- :/
# Cleanup backups when satisfied:
# find config -type f -name '*.bak' -delete

---

Handy OneLiners Recap

• Replace path everywhere (safe delimiter):

  perl -pi.bak -e 's|/var/www|/srv/www|g' $(git ls-files '*.conf')
  

• Insert block after header:

  perl -0777 -pe 's/(\[logging\]\n)/$1level="info"\nformat="json"\n/s' -i.bak /etc/app.ini
  

• Structured JSON tweak:

  jq '.service.enabled=true' config.json | sponge config.json
  

• YAML port update:

  yq -i '.server.port=8081' config.yaml
  

• INI using crudini:

  crudini --set /etc/app.ini server port 8081
  

---

Pro Tip: For customerfacing SOPs, turn these into parameterized scripts or Ansible roles. It's safer, faster, and easier to audit.

Further help for your team & clients:
Knowledgebase: https://www.domainindia.com/knowledgebase
Submit a Ticket: https://www.domainindia.com/support