Over the last three years, I have worked with my teams, stakeholders, end-users, and antivirus (a/v) and endpoint detection response(EDR) account reps to help secure my company's assets. During this journey, one common thread is that marketing teams must be explicit about what exact SKUs these companies offer. The only way to find out what companies offer is to contact sales and have a conversation. It would be best to look at each platform's features, write down the features you think you would like, and ask how their pricing bundles work. This will allow you to determine if their "Enterprise" SKU includes all the features you need versus the ones they are trying to sell you.
The guide below is material I created and have utilized over the last few years when evaluating EDRs. While it is not an all-encompassing list of items, the items are a great building block to launch your version of an EDR POC. With that said, check out the resources below.
Questions to ask an EDR vendor before you POC their product.
- How does your EDR detect and respond to threats?
- Can you provide examples of threats your EDR has detected and responded to?
- Does the EDR integrate with other security tools, such as firewalls and intrusion detection systems?
- How easy is it to use and manage the EDR?
- How scalable is the EDR? Can it accommodate a large number of endpoints?
- What kind of technical support and updates do you offer?
- How does your EDR help with compliance and regulatory requirements?
- How does your EDR handle false positives?
- Can you provide a demo or trial of the software?
- What are the pricing options, and what is included in the package?
- How does your EDR handle data privacy and security?
- How does the EDR handle remote workers and mobile devices?
- Does the EDR provide threat intelligence and analytics?
- Does the EDR have an incident response plan and incident management?
- Is SSO based on service tier?
- Is console access IP based?
- What IAM roles are available on your platform?
While these may seem like many questions, it's essential to find out this information to determine the maturity of each company and the platform.
Below is a Comparison Chart I developed in Notion, but it can easily be a Confluence page, Google Sheets, or Excel Worksheet.
Here is an EDR & XDR Comparison Chart.
My advice to all of you is to choose 2 - 3 platforms for POC. You can evaluate and resource as many as you would like, but only select one of the tools you think will be good for your environment and company. Preparation is critical, and before you gain access to your POC environments, please set up your testing environment for each POC.
- Set up 3-5 of each Operating System for each tool you are testing. Prep the physical machines, virtual machines, or remote desktops before getting access to tenants.
- Identify Beta users who will enroll their devices into the POC platforms.
If you use Beta users, there are two routes you can take, fully automate the installation experience or have users manually install the agent. The are pros and cons to both; however, you should utilize whatever will help enroll users into your program with the least amount of friction. Some environments are more technical than others, so choose wisely.
Key things to complete before Kicking off your POC.
- Set a timeline for the start and stopping of the POC
- Prep Machines for POCs
- Prep documentation for user enrollment & unenrollment
- Prep documentation for why the company is evaluating an EDR tool.
- Prep what information will be collected during the POC
- Identity Beta Users
- Identify outspoken users 1 - 2 minimum
- Identify proponents of IT / 1 - 2 people minimum
- Identify Engineers/Technical individuals
- Identify a Senior Member of the executive team
Here is a sample EDR Testing plan you could utilize as a framework. Please note that this should only be a framework, and you need to modify this to meet your organizational needs based on the SKUs you are acquiring. For example, if you pick up a Vulnerability Management plugin from your EDR vendor, you will need to add in testing to verify the module's functionality. EDR & XDR Testing Plan.
EDR (Endpoint Detection and Response) and XDR (Extended Detection and Response) Test Plan
Introduction: This test plan outlines the testing process for evaluating the effectiveness of an EDR and XDR solution. The goal of this testing is to ensure that the EDR and XDR solution can detect, respond to, and prevent security threats on endpoint devices and networks.
Scope: The scope of this test plan includes the following:
- Evaluating the EDR and XDR solution's ability to detect, respond to, and prevent security threats on endpoint devices and across the network.
- We are evaluating the EDR and XDR solution's ability to integrate with other security solutions, such as firewalls, intrusion detection and prevention, and antivirus.
- We are evaluating the EDR and XDR solution's ability to provide detailed forensic information on security threats and incidents.
- Evaluating the EDR and XDR solution's ease of use and management.
Pre-Test Preparation:
- Configure the EDR and XDR solution according to the vendor's instructions.
- Set up test endpoints, including Windows and MacOS systems and mobile devices. Install the EDR and XDR agent on the test endpoints.
- Set up EDR and XDR solution test cases to detect and respond to.
- Create a test environment that simulates a production environment.
Testing Procedures:
Threat Detection and Response Testing:
- Inject test cases of known security threats, such as malware, into the test environment.
- Observe the EDR and XDR solution's ability to detect and respond to threats.
- Evaluate the EDR and XDR solution's ability to prevent the threats from executing.
- Evaluate the EDR and XDR solution's ability to provide detailed forensic information on the threats and incidents.
Integration Testing:
- Test the EDR and XDR solution's ability to integrate with other security solutions, such as firewalls, intrusion detection and prevention, and antivirus.
- Test the EDR and XDR solution's ability to share threat intelligence with other security solutions.
- Evaluate the EDR and XDR solution's ability to automate incident response actions across multiple security solutions.
Ease of Use and Management Testing:
Test the EDR and XDR solution's ease of use and management, including the ability to:
- Configure and deploy the EDR and XDR agent on endpoint devices.
- Manage and monitor the EDR and XDR solution from a central console.
- Generate reports and alerts on security threats and incidents.
Acceptance Criteria:
- The EDR and XDR solution should detect and respond to at least 90% of the test cases of known security threats.
- The EDR and XDR solution should integrate with other security solutions and share threat intelligence with at least 80% accuracy.
- The EDR and XDR solution should be easy to use and manage, with a user-friendly interface and minimal training required.
Documentation:
- Test results, including screenshots, logs, and reports, should be documented and shared with the vendor for review and feedback.
- Any issues or bugs discovered during testing should be reported to the vendor and tracked until resolved.
- A final report should summarize the test results and include improvement recommendations.
Post-Test Clean-up:
- Remove the EDR and XDR solution from the test environment. Remove the EDR and XDR agents from the test endpoints.
- Delete any test cases.
If you need to test an EDR detection capability, I will use the eicar.org Anti-malware file. This file can be downloaded onto devices, resulting in a positive hit in an EDR system. If this fails to hit, I would ask EDR if they have any commands that will produce a positive result so you can simulate a positive/negative event.
Notion Database Tools
The links below will allow you to duplicate the database into your own instance of notion, and does allow for comments. I will follow up with the comments as well.
- EDR/NGAV Comparison Chart
- EDR/NGAV Testing Database
- EDR (Endpoint Detection and Response) and XDR (Extended Detection and Response) Test Plan
This post will provide the kicking-off point for getting your EDR testing and evaluation off the ground.
Please contact me if anyone has any comments or wants to add to the samples provided.
