EC2 SSH vs AWS SSM Session Manager – Secure Access Lab (Step-by-Step)

Introduction

If you’ve ever SSH’d into an EC2 instance using port 22 and a .pem key file, you’ve done what every AWS beginner does. It works, it’s straightforward, and it gets you into your instance quickly. But here’s the thing—in real enterprise environments, that approach often gets you a stern conversation with the security team.

This hands-on AWS lab teaches you two methods of connecting to your EC2 instance: traditional SSH and AWS Systems Manager Session Manager (SSM). More importantly, you’ll understand why SSM has become the preferred method in regulated industries and security-conscious organizations.

Who is this lab for? If you’re preparing for the AWS Solutions Architect exam, working toward a DevOps role, or simply want to understand how production AWS environments actually operate, this lab is essential. I’ve designed it for engineers who are comfortable launching EC2 instances but haven’t yet explored keyless, auditable access patterns.

The architect’s perspective: I’ve worked on environments where a single exposed port 22 triggered a compliance audit failure. I’ve also seen teams waste hours troubleshooting SSH connectivity when their Security Groups, NACLs, and key pairs were all misconfigured simultaneously. SSM eliminates most of these headaches while giving you full audit trails that security teams actually appreciate.

Common mistakes I see: Beginners often leave port 22 open to 0.0.0.0/0 because “it’s just a dev instance.” Intermediate engineers sometimes configure SSM but forget the IAM role attachment, then assume SSM itself is broken. Both groups underestimate how much simpler operations become when you remove SSH from the equation entirely.

Lab Overview

In this step-by-step AWS tutorial, you’ll accomplish the following:

You’ll verify that your EC2 instance meets SSM prerequisites, then connect using traditional SSH with a key pair. After that, you’ll configure AWS Systems Manager and connect using Session Manager—without opening any inbound ports. Finally, you’ll compare both experiences and understand when each method makes sense.

Skills you’ll gain: Secure EC2 access configuration, IAM role attachment for SSM, SSH key pair management, Session Manager connectivity, and troubleshooting both access methods.

Real-world enterprise use cases: This knowledge becomes critical during SOC2 audits when auditors ask how you control instance access. It matters for PCI-DSS compliance where logging every session is mandatory. ISO 27001 environments require demonstrable access controls—SSM provides exactly that. When a security incident occurs, having SSM session logs can be the difference between a quick resolution and weeks of forensic guesswork.

Prerequisites

Before starting this hands-on lab, ensure you have the following ready:

AWS Account: You need an active AWS account with permissions to create and manage EC2 instances, IAM roles, and Systems Manager resources. A personal account or sandbox environment works perfectly.

Existing EC2 Instance: Launch an Amazon Linux 2023 or Amazon Linux 2 instance if you don’t have one running. These AMIs come with the SSM Agent pre-installed, which saves configuration time.

Key Pair: You should have the .pem file for the key pair associated with your EC2 instance. If you’ve lost it, you won’t be able to SSH—but SSM will still work once configured, which actually demonstrates one of SSM’s advantages.

IAM Permissions: Your AWS user or role needs permissions for ec2:*, ssm:*, and iam:PassRole. For production, you’d scope these down significantly, but for learning purposes, broader permissions prevent permission-related troubleshooting distractions.

SSM Agent Requirements: The SSM Agent must be installed and running on your instance. Amazon Linux 2 and Amazon Linux 2023 include it by default. Ubuntu, RHEL, and Windows instances may require manual installation.

AWS CLI (Optional but Recommended): Having AWS CLI configured locally lets you initiate SSM sessions from your terminal, which mirrors how many DevOps engineers work daily.

Step-by-Step Hands-On Lab

Step 1: Verify EC2 Instance Prerequisites

Before attempting any connection, let’s confirm your instance is properly configured for both SSH and SSM access.

What to do: Navigate to the EC2 Console → Instances → Select your instance → Review the Details tab.

Verify these items:

Check that your instance has a public IP address assigned (required for SSH from outside AWS). Note the instance ID—you’ll need it for SSM connections. Confirm the instance state shows “Running” with 2/2 status checks passed.

For SSM specifically: Look at the IAM role attached to your instance. If the “IAM Role” field is empty, SSM won’t work regardless of any other configuration. We’ll fix this in Step 4.

Why this matters: I’ve seen engineers spend hours troubleshooting connectivity when their instance simply didn’t have a public IP, or when status checks were failing due to underlying hardware issues. Always verify the basics first.

Step 2: Connect to EC2 Using SSH

Traditional SSH requires three things working together: your key pair, your Security Group allowing port 22, and network connectivity to the instance’s public IP.

Security Group Configuration:

Navigate to EC2 Console → Security Groups → Select the security group attached to your instance.

Verify an inbound rule exists with:

• Type: SSH
• Protocol: TCP
• Port: 22
• Source: Your IP address (use “My IP” in the console, never 0.0.0.0/0 in production)

The SSH Command:

Open your terminal and run:

# Ensure your key has proper permissions first
chmod 400 your-key-pair.pem

# Connect to your instance
ssh -i your-key-pair.pem ec2-user@<public-ip-address>

Replace ec2-user with ubuntu for Ubuntu AMIs or admin for Debian.

What you should see: After accepting the host fingerprint, you’ll land at a shell prompt like [ec2-user@ip-10-0-1-45 ~]$. You’re now inside your EC2 instance.

Common misconfigurations: The most frequent SSH failure I encounter is permission errors on the .pem file. AWS rejects connections if your key file has overly permissive access. The chmod 400 command fixes this. Second most common: engineers SSH to the private IP instead of the public IP when connecting from outside AWS.

Step 3: Understanding SSH Limitations

While you’re connected via SSH, consider what this connection requires:

Port 22 must be open in your Security Group. That’s an inbound rule—a potential attack vector. Your key pair must be securely stored and distributed to anyone who needs access. There’s no native audit trail of what commands were executed during the session. Key rotation requires coordinating new .pem files across all users. If you lose the key, you lose access (unless you’ve configured alternative authentication).

I once worked with a team that had 47 different developers SSH into production instances, each with their own key pair. When an engineer left the company, they had to rotate keys across dozens of instances. It took three days and caused two outages. SSM would have prevented that entire situation.

Step 4: Enable and Configure AWS Systems Manager

Now let’s configure SSM access, which requires no inbound ports whatsoever.

Create the IAM Role:

Navigate to IAM Console → Roles → Create Role.

Select “AWS Service” as the trusted entity type, then choose “EC2” as the use case.

Search for and attach the AmazonSSMManagedInstanceCore policy. This policy grants the minimum permissions your instance needs to communicate with Systems Manager.

Name your role something descriptive like EC2-SSM-Role and create it.

Attach the Role to Your Instance:

Go back to EC2 Console → Instances → Select your instance → Actions → Security → Modify IAM Role.

Select the role you just created and save.

Why this matters: The IAM role allows your EC2 instance to communicate outbound to the Systems Manager service endpoints. No inbound rules needed. The instance registers itself with Systems Manager, and you connect through AWS’s control plane—never directly to the instance.

What you should see: Wait 2-3 minutes after attaching the role. Then navigate to Systems Manager Console → Fleet Manager (or Session Manager → Sessions). Your instance should appear in the managed instances list.

Step 5: Connect to EC2 Using Session Manager

Console Method:

Navigate to Systems Manager Console → Session Manager → Start Session.

Select your instance from the list and click “Start Session.”

A browser-based terminal opens. You’re now connected to your instance without SSH, without port 22, and without any key pair.

CLI Method:

If you have AWS CLI configured:

aws ssm start-session --target i-0abc123def456789

Replace the instance ID with your own. This opens an interactive session in your terminal.

What you should see: A shell prompt appears, typically as ssm-user rather than ec2-user. You have the same access to run commands, install packages, and troubleshoot issues.

Step 6: Compare SSH vs SSM Behavior

Run these commands in both sessions to observe the differences:

whoami
pwd
sudo -i

With SSH, you log in as ec2-user (or your AMI’s default user). With SSM, you log in as ssm-user but can still escalate to root. Both give you full instance access, but SSM sessions are automatically logged to CloudWatch Logs and S3 if you configure it—every keystroke, auditable.

Real Lab Experiences (Architect Insights)

Let me share some battle-tested lessons from production environments.

SSH mistakes that caused real problems: A team left port 22 open to the internet on a “temporary” dev instance. Within six hours, automated bots were attempting brute-force attacks. The instance wasn’t compromised, but the security alert it triggered consumed an entire day of incident response time.

Why port 22 becomes a liability: Every open port is an attack surface. Security scanners flag it. Compliance auditors question it. Penetration testers target it. Even if you restrict it to specific IPs, those IPs change when engineers work from home or travel.

IAM and SSM misconfigurations: The most common SSM issue I troubleshoot is the missing IAM role. Engineers install the SSM Agent, configure everything else perfectly, but forget that the instance itself needs IAM permissions to reach Systems Manager endpoints. Second most common: instances in private subnets without NAT Gateways or VPC endpoints can’t reach Systems Manager, so they never appear as managed instances.

Advice for junior DevOps engineers: Start with SSM as your default. Only configure SSH when you have a specific requirement that SSM can’t meet. When auditors ask about instance access controls, you’ll have clean, centralized answers instead of explaining key pair distribution across your team.

Validation & Testing

SSH Verification:

# From your local machine
ssh -i your-key.pem ec2-user@<public-ip> "echo 'SSH connection successful'"

Expected output: SSH connection successful

SSM Verification:

# Check if instance is registered with Systems Manager
aws ssm describe-instance-information --query 'InstanceInformationList[*].[InstanceId,PingStatus]' --output table

Expected output: Your instance ID with Online status.

Session Test:

aws ssm start-session --target i-0abc123def456789 --document-name AWS-StartInteractiveCommand --parameters command="hostname"

Expected output: Your instance’s hostname printed to the terminal.

Troubleshooting Guide

SSH connection refused: First, verify port 22 is open in your Security Group. Then check that your instance has a public IP and that you’re using the correct one. Run ssh -v for verbose output that reveals where the connection fails.

SSM session not appearing: Run these commands on the instance (you’ll need SSH access or EC2 Instance Connect for initial access):

# Check SSM Agent status
sudo systemctl status amazon-ssm-agent

# View SSM Agent logs
sudo journalctl -u amazon-ssm-agent -f

# Verify instance can reach SSM endpoints
curl -s https://ssm.us-east-1.amazonaws.com

IAM role missing permissions: Verify the role is attached to your instance and includes AmazonSSMManagedInstanceCore:

# From the instance
curl http://169.254.169.254/latest/meta-data/iam/security-credentials/

If this returns nothing or errors, the IAM role isn’t properly attached.

Instance metadata issues: Some hardened AMIs block IMDS access, which breaks SSM:

curl -w "%{http_code}" http://169.254.169.254/latest/meta-data/

You should get a 200 response. 401 or 403 indicates IMDS restrictions.

Region mismatches: SSM console shows instances per-region. Ensure you’re viewing the correct region in both the EC2 and Systems Manager consoles.

AWS Best Practices (Solutions Architect Level)

Security (Zero-Trust, Keyless Access): Eliminate SSH keys entirely in production. Use SSM for all instance access. Implement IAM policies that restrict who can start sessions and on which instances. Enable session logging to S3 and CloudWatch.

Reliability: SSM depends on the agent running and IAM roles being correctly configured. Include SSM Agent health checks in your instance monitoring. Consider using EC2 Instance Connect as a backup access method.

Operational Excellence: Tag instances consistently so you can identify them in Session Manager. Use SSM documents to standardize administrative tasks. Implement break-glass procedures for emergency access that don’t rely on SSH keys.

Cost Optimization: SSM Session Manager is free. You pay only for optional logging to S3/CloudWatch. Compare this to maintaining bastion hosts for SSH access—SSM eliminates that infrastructure cost entirely.

Performance Efficiency: SSM sessions have slightly higher latency than direct SSH due to the AWS control plane routing. For most administrative tasks, this is imperceptible. For high-throughput file transfers, consider AWS DataSync or S3.

Logging & Auditing: Enable session activity logging in Session Manager preferences. Configure CloudTrail to capture StartSession API calls. This combination provides complete audit trails that satisfy most compliance frameworks.

Tagging Strategy: Use tags like Environment, Application, and Owner on instances. These tags appear in Session Manager, making it easy to identify the correct instance before starting a session.

Interview Questions: EC2 SSH and SSM Session Manager

If you’re preparing for AWS Solutions Architect, DevOps Engineer, or Cloud Engineer interviews, expect questions about secure instance access. Here are real questions I’ve asked candidates and been asked myself, organized by experience level.

Junior Level Questions

Q1: What is the difference between SSH and AWS Systems Manager Session Manager?

What interviewers want to hear: SSH is a network protocol that requires port 22 to be open in your Security Group and a key pair for authentication. Session Manager is an AWS service that allows shell access through the AWS control plane without any inbound ports. SSM uses IAM for authentication instead of SSH keys, and it provides built-in session logging. Strong candidates mention that SSM eliminates the need to manage and distribute key pairs across teams.

Q2: What port does SSH use, and why is opening it considered a security risk?

What interviewers want to hear: SSH uses TCP port 22 by default. Opening port 22 creates an attack surface because automated bots constantly scan for open SSH ports to attempt brute-force attacks. Even with key-based authentication, the open port itself generates security alerts and can trigger compliance audit findings. Good answers mention that restricting the source IP helps but doesn’t eliminate the risk entirely.

Q3: What happens if you lose your EC2 key pair file?

What interviewers want to hear: You cannot recover a lost .pem file from AWS—there’s no download option after initial creation. Without the key pair, traditional SSH access is impossible. Recovery options include using EC2 Instance Connect, Session Manager (if configured), stopping the instance and mounting the volume to another instance to add a new public key, or using EC2 Serial Console. Candidates who mention SSM as a backup access method demonstrate practical thinking.

Mid-Level Questions

Q4: Walk me through how Session Manager establishes a connection without inbound ports.

What interviewers want to hear: The SSM Agent on the EC2 instance maintains an outbound HTTPS connection to the Systems Manager service endpoints. When you start a session, your request goes to Systems Manager (authenticated by your IAM credentials), and the service routes your session through the existing outbound connection from the instance. There’s never a direct network path from your machine to the instance. Strong candidates mention that this works even for instances in private subnets if they have NAT Gateway access or VPC endpoints for Systems Manager.

Q5: What IAM policy is required for an EC2 instance to use Session Manager?

What interviewers want to hear: The instance needs an IAM role with the AmazonSSMManagedInstanceCore managed policy attached. This policy grants permissions for the SSM Agent to communicate with Systems Manager endpoints, send heartbeats, and retrieve commands. For production, candidates might mention creating custom policies with more restrictive permissions or adding AmazonSSMManagedEC2InstanceDefaultPolicy for additional features.

Q6: An instance isn’t appearing in Session Manager’s managed instances list. How would you troubleshoot this?

What interviewers want to hear: A structured troubleshooting approach: First, verify the IAM role is attached to the instance and includes SSM permissions. Second, check if the SSM Agent is running using systemctl status amazon-ssm-agent. Third, verify network connectivity—the instance needs outbound HTTPS access to SSM endpoints (either through internet access or VPC endpoints). Fourth, check the agent logs with journalctl -u amazon-ssm-agent. Fifth, confirm you’re looking at the correct AWS region. Strong candidates mention checking instance metadata access with curl http://169.254.169.254/latest/meta-data/iam/security-credentials/.

Q7: How do you enable session logging for compliance and auditing purposes?

What interviewers want to hear: In the Session Manager preferences, you can configure logging to CloudWatch Logs, S3, or both. You specify the log group or S3 bucket, and optionally enable encryption with KMS. Once enabled, every command typed during sessions is logged with timestamps and user identity. For compliance frameworks like SOC2 and PCI-DSS, candidates should mention that CloudTrail also captures the StartSession API calls, providing a complete audit trail of who accessed which instances and when.

Senior/Architect Level Questions

Q8: Design a secure access strategy for EC2 instances in a regulated environment where SSH is prohibited.

What interviewers want to hear: A comprehensive answer covers multiple layers. Primary access through Session Manager with IAM policies restricting which users can access which instances (using tags or instance IDs in conditions). Session logging enabled to encrypted S3 buckets with lifecycle policies for retention. VPC endpoints for Systems Manager in private subnets to avoid internet traversal. CloudTrail enabled for API-level auditing. Break-glass procedures documented for emergency access. Integration with identity federation (SSO) so session access ties to corporate identity. Strong candidates also mention using Systems Manager Run Command for automated tasks instead of interactive sessions where possible.

Q9: Compare the security implications of SSH bastion hosts versus Session Manager for a multi-account AWS environment.

What interviewers want to hear: Bastion hosts require maintaining additional infrastructure (patching, monitoring, high availability), managing SSH keys across accounts, and opening port 22 between VPCs or accounts. They create a single point of failure and a high-value target for attackers. Session Manager eliminates bastion infrastructure entirely, centralizes access through IAM (which can federate with corporate identity), provides native audit logging, and requires no cross-account network connectivity since sessions route through each account’s regional SSM endpoints. The security boundary shifts from network perimeter to IAM policies. Cost-wise, SSM eliminates bastion host EC2 costs and operational overhead.

Q10: How would you implement least-privilege access for Session Manager across different teams?

What interviewers want to hear: Use IAM policies with conditions that restrict ssm:StartSession to specific instances. You can use ssm:resourceTag conditions to allow teams access only to instances tagged with their team name. For example, the DevOps team can only start sessions on instances tagged Team=DevOps. Combine this with AWS Organizations SCPs to enforce these restrictions across accounts. Session Manager also supports document-based access controls where you can restrict users to specific session documents that limit their capabilities (for example, a document that only allows read-only commands). Strong candidates mention combining this with IAM Identity Center for centralized user management.

Frequently Asked Questions (FAQs)

These questions address the most common searches around EC2 access methods. Each answer is structured for clarity and optimized for Google’s featured snippet format.

Conclusion + Next Lab Recommendation

You’ve now connected to an EC2 instance using both traditional SSH and AWS Systems Manager Session Manager. More importantly, you understand why SSM has become the enterprise standard for secure EC2 access.

SSH requires open ports, key management, and provides no native auditing. SSM requires no inbound ports, uses IAM for authentication, and logs every session automatically. For regulated environments—and honestly, for any environment that takes security seriously—SSM is the clear choice.

What you achieved: You verified instance prerequisites, connected via SSH, configured IAM roles for SSM, connected via Session Manager, and learned troubleshooting techniques for both methods.

Why SSM is the future: As organizations adopt zero-trust security models, any access method that requires open inbound ports becomes increasingly difficult to justify. SSM aligns with modern security practices while simplifying operations.

Next up: In Lab 3 — EC2 Security Groups Deep Dive {link}, you’ll master Security Group design patterns, understand stateful vs. stateless rules, and implement defense-in-depth configurations that pass security audits. The knowledge you gained today about SSH port requirements makes that lab directly relevant.

Related Posts:

• Lab 1 — Launching Your First EC2 Instance
• Understanding IAM Roles for EC2
• AWS Systems Manager Complete Guide

External Resources:

• AWS Systems Manager Session Manager Documentation
• EC2 Key Pairs Documentation
• IAM Roles for EC2