mtc-project-sync-script/README.md

# MTC Project Sync Script

A comprehensive automation toolkit for synchronizing and managing projects between remote servers and local development environments. Designed for developers working with remote servers who need efficient project setup and bidirectional file synchronization.

## Table of Contents

- [Overview](#overview)
- [Features](#features)
- [Requirements](#requirements)
- [Installation](#installation)
- [Quick Start](#quick-start)
- [Scripts Reference](#scripts-reference)
  - [sync-project.sh](#sync-projectsh)
  - [local-remote.sh](#local-remotesh)
  - [remote-local.sh](#remote-localsh)
  - [sftp-watch.sh](#sftp-watchsh)
  - [build-watch.sh](#build-watchsh)
- [Configuration](#configuration)
- [Troubleshooting](#troubleshooting)
- [License](#license)

## Overview

**MTC Project Sync Script** is a bash-based automation suite that streamlines remote project development workflows. It handles initial project setup, ongoing file synchronization, and integrates seamlessly with VS Code for SFTP-based development.

### What It Does

- Downloads complete projects from remote servers with one command
- Configures VS Code SFTP settings automatically
- Provides helper scripts for continuous sync and build workflows
- Manages bidirectional file synchronization between local and remote environments
- Integrates lazygit for convenient version control

## Features

- **One-Command Setup** - Initialize and download remote projects instantly
- **VS Code Integration** - Auto-generates SFTP configuration with optimized ignore patterns
- **Bidirectional Sync** - Push local changes to server or pull remote changes to local
- **Watch Mode Support** - Automatically sync files on changes
- **Build Integration** - Watch and upload build artifacts automatically
- **Error Handling** - Comprehensive validation at each step
- **Clean Architecture** - Modular scripts for different workflows
- **Version Control Ready** - Includes lazygit and proper .gitignore setup

## Requirements

### System Dependencies

- **Bash** (4.0 or higher)
- **SSH/SCP** - Secure shell and copy utilities
- **sshpass** - Password-based SSH authentication
- **tar** - Archive utility
- **rsync** - Fast file synchronization tool
- **watchexec** (optional) - For watch mode functionality

### Installation of Dependencies

#### macOS (using Homebrew)
```bash
brew install sshpass rsync watchexec
```

#### Ubuntu/Debian
```bash
sudo apt-get install sshpass rsync openssh-client
# For watch functionality:
cargo install watchexec-cli
# or download from: https://github.com/watchexec/watchexec/releases
```

#### CentOS/RHEL
```bash
sudo yum install sshpass rsync openssh-clients
```

### Server Requirements

- SSH access with username/password or key-based authentication
- Sufficient permissions to create/read files in project directory
- tar utility installed on remote server

## Installation

### Method 1: Clone Repository

```bash
cd ~/Documents/GitHub
git clone <repository-url> mtc-project-sync-script
cd mtc-project-sync-script
chmod +x sync-project.sh
```

### Method 2: Create Symbolic Link (Recommended)

For easy access from any project directory:

```bash
# From your project directory
ln -s ~/Documents/GitHub/mtc-project-sync-script/sync-project.sh sync-project.sh

# Run the script
./sync-project.sh
```

This creates a symbolic link that allows you to use the script without copying it to every project.

## Quick Start

1. **Navigate to your desired project directory:**
   ```bash
   mkdir my-project && cd my-project
   ```

2. **Create symbolic link or copy the script:**
   ```bash
   ln -s ~/Documents/GitHub/mtc-project-sync-script/sync-project.sh .
   ```

3. **Run the sync script:**
   ```bash
   ./sync-project.sh
   ```

4. **Provide the requested information:**
   - Server address (e.g., `username@server.com`)
   - Server project path (e.g., `/home/username/public_html`)
   - Local folder path (e.g., `./project-name`)
   - SSH password when prompted

5. **Start developing!** The script will have:
   - Downloaded all project files
   - Created VS Code SFTP configuration
   - Generated helper scripts for ongoing development
   - Set up lazygit for version control

## Scripts Reference

### sync-project.sh

**Main synchronization script** - Downloads a complete project from a remote server and configures the local development environment.

#### Usage
```bash
./sync-project.sh
```

#### What It Does

1. **Prompts for Configuration:**
   - Server credentials (user@host)
   - Remote project path
   - Local destination folder
   - SSH password

2. **Creates Archive on Server:**
   - Compresses the entire project directory into `leo.tar.gz`
   - Stores archive in parent directory temporarily

3. **Downloads Archive:**
   - Uses SCP to securely transfer the archive
   - Shows progress during transfer

4. **Extracts Locally:**
   - Unpacks the archive to specified local folder
   - Preserves file permissions and structure

5. **Generates VS Code Configuration:**
   - Creates `.vscode/sftp.json` with:
     - Server connection details
     - Auto-upload on save enabled
     - Optimized ignore patterns

6. **Creates Helper Scripts:**
   - Generates `local-remote.sh`, `remote-local.sh`, `sftp-watch.sh`, and `build-watch.sh`
   - Pre-configures them with your server credentials
   - Adds them to `.gitignore`

7. **Deploys lazygit:**
   - Copies lazygit binary to local project
   - Uploads lazygit to remote server
   - Adds to `.gitignore`

8. **Cleanup:**
   - Removes temporary archive files from both local and remote

#### Example Session
```bash
$ ./sync-project.sh

=== Project Sync Configuration ===

Server user@host (e.g., leo@server.com): john@myserver.com
Server project path (e.g., /home/leo/public_leo): /home/john/websites/shop
Local folder path (e.g., Delta Pharmacy): online-shop
SSH Password: [hidden]

Starting project sync...
Creating archive on server...
Archive created successfully
Copying archive to local machine...
Archive copied successfully
Extracting archive...
Archive extracted successfully
Cleaning up local archive...
Cleaning up server archive...
File sync complete!
.vscode/sftp.json created successfully
build-watch.sh created successfully
Added build-watch.sh to .gitignore
local-remote.sh created successfully
Added local-remote.sh to .gitignore
remote-local.sh created successfully
Added remote-local.sh to .gitignore
sftp-watch.sh created successfully
Added sftp-watch.sh to .gitignore
Uploading lazygit to server...
lazygit uploaded successfully
Added lazygit to .gitignore
```

---

### local-remote.sh

**Push local changes to remote server** - Synchronizes files from your local machine to the remote server.

#### Usage
```bash
./local-remote.sh
```

#### What It Does

- Uses `rsync` to upload local files to remote server
- Preserves timestamps and permissions
- Deletes files on server that don't exist locally (using `--delete` flag)
- Excludes common development directories (node_modules, vendor, storage, etc.)
- Only uploads changed files for efficiency

#### Excluded Directories
- `.git`
- `.vscode`
- `.DS_Store`
- `node_modules`
- `vendor`
- `storage`
- `uploads`
- `temp`
- `cache`
- `sitepress-multilingual-cms`

#### Use Cases
- Deploy local changes to staging/production
- Update server after local development session
- Restore server files from local backup

---

### remote-local.sh

**Pull remote changes to local** - Synchronizes files from the remote server to your local machine.

#### Usage
```bash
./remote-local.sh
```

#### What It Does

- Uses `rsync` to download files from remote server
- Preserves timestamps and permissions
- Does NOT delete local files (no `--delete` flag)
- Excludes common development directories
- Only downloads changed files for efficiency

#### Excluded Directories
- `.vscode`
- `.DS_Store`
- `node_modules`
- `vendor`
- `storage`
- `uploads`
- `temp`
- `cache`
- `sitepress-multilingual-cms`

#### Use Cases
- Update local environment with server changes
- Pull production data for debugging
- Sync changes made by other team members
- Backup server files to local machine

---

### sftp-watch.sh

**Bidirectional sync with watch mode** - Continuously monitors file changes and syncs automatically.

#### Usage
```bash
./sftp-watch.sh
```

#### What It Does

1. **Initial Sync:**
   - Runs `remote-local.sh` first to ensure local is up-to-date
   - Displays "Syncing remote->local..." message

2. **Watch Mode:**
   - Monitors current directory for file changes
   - Debounces changes (500ms delay) to avoid excessive syncing
   - Automatically runs `local-remote.sh` on any file modification

#### Requirements
- Requires `watchexec` to be installed

#### Use Cases
- Development with immediate server deployment
- Real-time collaboration on remote server
- Continuous integration workflows
- Testing on remote environment while coding locally

#### Example Workflow
```bash
# Terminal 1: Start watch mode
./sftp-watch.sh

# Terminal 2: Make changes to your code
vim index.php
# Changes automatically sync to server after save
```

---

### build-watch.sh

**Watch and build mode** - Monitors source files, rebuilds on changes, and deploys build output.

#### Usage
```bash
./build-watch.sh
```

#### What It Does

1. Monitors current directory for file changes (2-second debounce)
2. Runs `npm run build` when changes are detected
3. Automatically executes `local-remote.sh` after successful build
4. Uploads built assets to remote server

#### Requirements
- Requires `watchexec` to be installed
- Requires `package.json` with build script defined
- Node.js and npm installed

#### Use Cases
- Frontend development with build steps (React, Vue, etc.)
- Asset compilation (Sass, Less, TypeScript)
- Continuous deployment of build artifacts
- Development where server runs production builds

#### Example Workflow
```bash
# Terminal: Start build watch
./build-watch.sh

# Make changes to source files
# Watches for changes -> Runs npm run build -> Uploads to server
```

#### package.json Example
```json
{
  "scripts": {
    "build": "webpack --mode production"
  }
}
```

---

## Configuration

### VS Code SFTP Configuration

The sync script automatically generates `.vscode/sftp.json`:

```json
{
  "name": "project-name",
  "host": "server.com",
  "protocol": "sftp",
  "port": 22,
  "username": "your-username",
  "password": "your-password",
  "remotePath": "/path/to/remote/project",
  "uploadOnSave": true,
  "useTempFile": false,
  "openSsh": false,
  "ignore": [
    ".vscode",
    ".DS_Store",
    "node_modules",
    "vendor",
    "storage",
    "uploads",
    "temp",
    "cache",
    "sitepress-multilingual-cms"
  ]
}
```

### Customizing Ignore Patterns

Edit `.vscode/sftp.json` or the helper scripts to add/remove excluded directories:

```bash
# Edit local-remote.sh or remote-local.sh
--exclude custom_folder \
```

### Security Considerations

1. **Credentials in Scripts:**
   - Helper scripts contain plaintext passwords
   - These scripts are automatically added to `.gitignore`
   - Never commit these files to version control

2. **Alternative: SSH Keys**
   - Consider setting up SSH key-based authentication
   - Eliminates need for password storage
   - More secure for production environments

   ```bash
   # Generate SSH key
   ssh-keygen -t rsa -b 4096
   
   # Copy to server
   ssh-copy-id user@server.com
   ```

## Troubleshooting

### Common Issues

#### "sshpass: command not found"
**Solution:** Install sshpass using your package manager (see [Requirements](#requirements))

#### "watchexec: command not found"
**Solution:** Install watchexec for watch mode functionality:
```bash
brew install watchexec  # macOS
cargo install watchexec-cli  # Cross-platform
```

#### "Permission denied" during SSH
**Solutions:**
- Verify username and password are correct
- Check that SSH is enabled on the server
- Ensure your user has permission to access the remote path
- Try connecting manually: `ssh user@server.com`

#### "Remote path not found"
**Solutions:**
- Verify the server project path exists
- Use absolute paths (e.g., `/home/user/project` not `~/project`)
- Check path spelling and case sensitivity

#### "Archive creation failed"
**Solutions:**
- Ensure you have write permissions in the parent directory
- Check if there's enough disk space on the server
- Verify tar is installed on the server

#### Files not syncing in watch mode
**Solutions:**
- Ensure watchexec is installed
- Check that you're editing files in the watched directory
- Try increasing debounce delay in the script
- Verify rsync is working manually

#### Build fails in build-watch.sh
**Solutions:**
- Verify `npm run build` works manually
- Check that Node.js and npm are installed
- Ensure package.json has a build script defined
- Review npm error messages for missing dependencies

### Debug Mode

Enable verbose output for rsync commands:

```bash
# Edit local-remote.sh or remote-local.sh
# Add -v flag to rsync
rsync -avzuv ...
```

### Manual Testing

Test components individually:

```bash
# Test SSH connection
ssh user@server.com

# Test SCP transfer
scp testfile.txt user@server.com:~/

# Test rsync
rsync -avzu --dry-run . user@server.com:~/test/
```

## Best Practices

1. **Initial Setup:**
   - Run `sync-project.sh` once per project
   - Keep the sync script in a central location and use symbolic links

2. **Development Workflow:**
   - Use `sftp-watch.sh` for rapid development with live server testing
   - Use `build-watch.sh` for projects with build steps
   - Use `local-remote.sh` for manual deployments

3. **Team Collaboration:**
   - Run `remote-local.sh` before starting work to get latest changes
   - Communicate with team when making large changes
   - Use version control (git) in parallel for code history

4. **Security:**
   - Never commit generated scripts with credentials
   - Use SSH keys for production servers
   - Limit server access with appropriate permissions

## License

This project is provided as-is for development and synchronization purposes.

## Contributing

Contributions, issues, and feature requests are welcome. Feel free to check the issues page or submit pull requests.

---

**Note:** This toolkit is designed for development workflows. Always test in staging environments before deploying to production servers.