Automatic downloads

1. Verify Deadline CLI Installation

   Get the exact path to your deadline executable:

   $ which deadline

   Note this path (e.g., /opt/homebrew/bin/deadline) for use in the plist file.

2. Create Checkpoint Directory

   Create the directory where checkpoint files will be stored. Ensure proper permissions for your user to run the command.

   $ mkdir -p /path/to/checkpoint/directory

3. Create Log Directory

   Create a directory for cron job logs:

   $ mkdir -p /path/to/logs

   Consider setting up log rotate on the log file using https://www.redhat.com/en/blog/setting-logrotate

4. Check Current Crontab

   View your current crontab to see existing jobs:

   $ crontab -l

5. Edit Crontab

   Open your crontab file for editing:

   $ crontab -e

   The first time you run the command, you might be prompted to choose an editor (nano, vim, and so on).

6. Add Cron Job Entry

   Add the following line to run the job every 5 minutes (replace paths with actual values from steps 1 and 2):

   */5 * * * * /path/to/deadline queue sync-output --profile deadline-downloader --farm-id YOUR_FARM_ID --queue-id YOUR_QUEUE_ID --storage-profile-id YOUR_PROFILE_ID --checkpoint-dir /path/to/checkpoint/directory >> /path/to/logs/deadline_sync.log 2>&1

7. Verify Cron Job Installation

   After saving and exiting the editor, verify the cron job was added:

   $ crontab -l

   You should see your new job listed.

8. Check Cron Service Status

   Ensure the cron service is running:

   # For systemd systems (most modern Linux distributions)
   $ sudo systemctl status cron
   # or
   $ sudo systemctl status crond
   
   # For older systems
   $ sudo service cron status

   If not running, start it:

   $ sudo systemctl start cron
   $ sudo systemctl enable cron  # Enable auto-start on boot

1. Verify Deadline CLI Installation

   Get the exact path to your deadline executable:

   $ which deadline

   Note this path (e.g., /opt/homebrew/bin/deadline) for use in the plist file.

2. Create Checkpoint Directory and Log Directory

   Create the directory where checkpoint files will be stored:

   $ mkdir -p /path/to/checkpoint/directory
   $ mkdir -p /path/to/logs

   Consider setting up log rotate on the log file using https://formulae.brew.sh/formula/logrotate

3. Create a Plist file

   Create a configuration file at ~/Library/LaunchAgents/com.user.deadlinesync.plist with the following content (replace /path/to/deadline with the actual path from step 1):

   <?xml version="1.0" encoding="UTF-8"?>
   <!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
   <plist version="1.0">
   <dict>
       <key>Label</key>
       <string>com.user.deadlinesync</string>
       <key>ProgramArguments</key>
       <array>
           <string>/path/to/deadline</string>
           <string>queue</string>
           <string>sync-output</string>
           <string>--profile</string>
           <string>deadline-downloader</string>
           <string>--farm-id</string>
           <string>YOUR_FARM_ID</string>
           <string>--queue-id</string>
           <string>YOUR_QUEUE_ID</string>
           <string>--storage-profile-id</string>
           <string>YOUR STORAGE PROFILE ID</string>
           <string>--checkpoint-dir</string>
           <string>/path/to/checkpoint/dir</string>
       </array>
       <key>RunAtLoad</key>
       <true/>
       <key>UserName</key>
       <string>YOUR_USER_NAME</string>
       <key>StandardOutPath</key>
       <string>/path/to/logs/deadline_sync.log</string>
       <key>StartInterval</key>
       <integer>300</integer>
   </dict>
   </plist>

   Replace --storage-profile-id YOUR_PROFILE_ID above with --ignore-storage-profiles if your downloading machine is the same as submitting machine.

4. Validate Plist File

   Validate the XML syntax of your plist file:

   $ plutil -lint ~/Library/LaunchAgents/com.user.deadlinesync.plist

   The command returns "OK" if the file is valid.

5. Check for Existing Launch Agents or Launch Daemons

   Check if a launch agent is already loaded:

   $ launchctl list | grep deadlinesync
   OR
   $ sudo launchctl list | grep deadlinesync

   If one exists, unload it first:

   $ launchctl bootout gui/$(id -u)/com.user.deadlinesync
   OR
   $ sudo launchctl bootout system/com.user.deadlinesync

6. Create and bootstrap

   To run this task while the user is logged in, run it as LaunchAgent. To run this task without a user being logged in every time the machine is running, run it as a LaunchDaemon.

   1. To run as LaunchAgent:

      1. Use the configuration created under ~/Library/LaunchAgents/com.user.deadlinesync.plist

      2. Then load the configuration using the bootstrap command:

         $ launchctl bootstrap gui/$(id -u) ~/Library/LaunchAgents/com.user.deadlinesync.plist

   2. To run as LaunchDaemon:

      1. Move the Pilst file and change permissions by running the following:

         $ sudo mv ~/Library/LaunchAgents/com.user.deadlinesync.plist /Library/LaunchDaemons/ 
         $ sudo chown root:wheel /Library/LaunchDaemons/com.user.deadlinesync.plist 
         $ sudo chmod 644 /Library/LaunchDaemons/com.user.deadlinesync.plist
         

      2. Load the launch agent using the modern bootstrap command:

         $ sudo launchctl bootstrap system /Library/LaunchDaemons/com.user.deadlinesync.plist

7. Verify Status

   If you bootstrapped a LaunchAgent run the following to confirm it's loaded:

   $ launchctl list | grep deadlinesync

   If you bootstrapped a LaunchDaemon, confirm it is loaded by running:

   $ sudo launchctl list | grep deadlinesync

   The output should look like

   SOME_PID_NUMBER 0 com.user.deadlinesync

   For detailed status information:

   $ launchctl print gui/$(id -u)/com.user.deadlinesync

   This shows the current state, program arguments, environment variables, run interval, and execution history.

1. Verify Deadline CLI Installation

   Find the deadline executable:

   C:\> where deadline

   Note the full path (e.g., C:\Program Files\Amazon\DeadlineCloud\deadline.exe) for use in the task.

2. Create Checkpoint Directory

   Create the directory where checkpoint files will be stored:

   C:\> mkdir "path\to\checkpoint\directory"

3. Create Log Directory

   Create a directory for task logs:

   C:\> mkdir "path\to\logs"

4. Create Batch File Wrapper

   Create the batch file with the following content:

   C:\> notepad C:\path\to\deadline_sync.bat

   YOUR_PATH_TO_DEADLINE.EXE queue sync-output --profile deadline-downloader --farm-id YOUR_FARM_ID --queue-id YOUR_QUEUE_ID --storage-profile-id YOUR_PROFILE_ID --checkpoint-dir path\to\checkpoint\checkpoints > path\to\logs\deadline.log 2>&1

5. Test Batch File

   Test the batch file manually:

   C:\> .\path\to\deadline_sync.bat

   Check the log file was created:

   C:\> notepad path\to\logs\deadline_sync.log

6. Check Task Scheduler Service

   Ensure Task Scheduler service is running:

   C:\> sc query "Schedule"

   If the service doesn't exist, try alternative names:

   C:\> sc query "TaskScheduler"
   C:\> sc query "Task Scheduler"

   If not running, start it:

   C:\> sc start "Schedule"

7. Create Scheduled Task

   Create the task to run every 5 minutes.

   C:\> schtasks /create /tn "DeadlineOutputSync" /tr "C:\path\to\deadline_sync.bat" /sc minute /mo 5

   Command breakdown:

   • /tn - Task name

   • /tr - Task to run (your batch file)

   • /sc minute /mo 5 - Schedule: every 5 minutes

8. Verify Task Creation

   Check that the task was created successfully:

   schtasks /query /tn "DeadlineOutputSync" /v /fo LIST

   Look for:

   • Task To Run: Should show your batch file path

   • Next Run Time: Should show a time within 5 minutes

9. Test Task Execution

   Run the task manually to test:

   schtasks /run /tn "DeadlineOutputSync"

   Check task status:

   schtasks /query /tn "DeadlineOutputSync"

1. Check if cron service is running:

   
   # For systemd systems
   $ sudo systemctl status cron
   # or
   $ sudo systemctl status crond
   
   # Check if your user has cron job correctly configured
   $ crontab -l
          

2. View cron execution logs:

   
   # Check system logs for cron activity (most common locations)
   $ sudo tail -f /var/log/syslog | grep CRON
   $ sudo tail -f /var/log/cron.log | grep deadline
   
   # View recent cron logs
   $ sudo journalctl -u cron -f
   $ sudo journalctl -u crond -f  # On some systems
          

3. Check your specific cron job logs:

   
   # View the log file specified in your cron job
   $ tail -100f /path/to/logs/deadline_sync.log
          

4. Search for cron job execution in system logs:

   
   # Look for your specific cron job executions
   $ sudo grep "deadline.*sync-output" /var/log/syslog
   
   # Check for cron job starts and completions
   $ sudo grep "$(whoami).*CMD.*deadline" /var/log/syslog
          

5. Check checkpoint file updates:

   
   # List checkpoint files with timestamps
   $ ls -la /path/to/checkpoint/directory/
   
   # Check when checkpoint was last modified
   $ stat /path/to/checkpoint/directory/queue-*_download_checkpoint.json
          

6. Check the log file:

   
   $ ls -la /path/to/log/deadline_sync.log
          

Viewing Launch Agent Execution Logs:

1. Check if the launch agent is running:

   
   $ sudo launchctl list | grep deadlinesync
          

   Output shows: PID Status Label (PID will be - when not currently running, which is normal for interval jobs)

2. View detailed launch agent status:

   
   $ sudo launchctl print system/com.user.deadlinesync
          

   This shows execution history, last exit code, number of runs, and current state.

3. View launch agent execution logs:

   
   # View recent logs (last hour)
   log show --predicate 'subsystem contains "com.user.deadlinesync"' --last 1h
   
   # View logs from a specific time period
   log show --predicate 'subsystem contains "com.user.deadlinesync"' --start '2024-08-27 09:00:00'
          

4. Force run the launch agent for immediate testing:

   
   $ sudo launchctl kickstart gui/$(id -u)/com.user.deadlinesync
          

   This immediately triggers the job regardless of the schedule, useful for testing.

5. Check checkpoint file updates:

   
   # List checkpoint files with timestamps
   $ ls -la /path/to/checkpoint/directory/
          

6. Check the log file:

   
   $ ls -la /path/to/log/deadline_sync.log
          

1. Check if Task Scheduler service is running:

   C:\> sc query "Schedule"

   If the service doesn't exist, try alternative names:

   C:\> sc query "TaskScheduler"
   C:\> sc query "Task Scheduler"

2. View your scheduled tasks:

   
   C:> schtasks /query /tn "DeadlineOutputSync"
          

3. Check your task's log file:

   
   # View the log file created by your batch script
   C:> notepad C:\path\to\logs\deadline_sync.log
          

4. Check checkpoint file updates:

   
   # List checkpoint files with timestamps
   C:> dir "C:\path\to\checkpoint\directory" /od