General Topics

• In the contract, the vendor needs to agree that you can use whatever tools and methods you think are available to an attacker to test its product.
• Independent certificationfor hosted service?
• Is your system susceptible to any SANS Top 20 security vulnerablities for Windows and UNIX described at http://www.sans.org/top20?
•      a. If so, which ones?
•      b. What is yuor timetable for correcting these vulnerabilities?
• What security-related certifications do those in your company who are involved with this porduct's development and support hold ? Examples of recognized certifications: SANS GSEC, CISSP, MCSE and CCIE.
• Are your developers trained in writing secure code?
• What is the best practice for locking down your product so that it is secure?
• Do we have direct, immediate access to a secure specialist within your company if a security problem should arise with your product?
• May we see a copy of your End User License Agreement?
• May we obtain an evaluation copy of your software in order to conduct a security assessment?  

•  

Cryptography

• Cryptographic components should use well-known and published strong ciphers.
• What ciphers do you use?
• Key lengths must be specified and aspects of key management must be specified. Local Administrator?
• Source of randomness must be specified.
• Unspecified components should be ignored and the cost of other solutions factored into the project cost, e.g. unspecified connection encryption would be ignored and the cost of providing an encrypted tunnel would be factored in; built into TCO.
• What encryption is used for data storage?
• What is used for session encryption?
• What industry standards and protocols such as SSL, TLS and LDAP are enacted within your system? What versions are supported?

PKI

• Can VT-CA certificates be used?
•      Are you using Java 1.4 vs. 1.5 ?

•      Can a CSR in the correct format be used?
•      Can you configure who the trusted CAs are?
•      Is the DN, etc. in a VT certificate usable?

Data formats

This particular topic is not traditionally considered a security topic. It is here because it represents an issue that can be very costly when it is time to replace a software product. The ability to compare various data or configuration snapshots is quite useful in incident analysis though.

Consider an application like Banner. The data are stored in Oracle tables which we can access with applications other than Banner. If we want to convert to some product, we know what data we have and where it is. If we want to generate reports not directly supported by Banner, we can do it with the normal database tools. Think about the
risk we would be taking if Banner stored the data in some proprietary format we couldn't directly access. Virtually all of our captive.

A second aspect of this issue is the usefulness of data over time. A 30-year old ASCII (or even EBCDIC) data file can still be used today. The same thing is true for a
LaTeX or nroff document. Even though formats may change over time, XML and associated tools will likely be around for a while and it provides a reasonable format for long-term data use.

• Is the format of the data files specified? I.e. could you write something to convert the data into something you can import into another product?
  • How to do it: ASCII text, CSV, XML (such as ODF), tables in a database
  • How not to do it: M$ Office
• Data includes configuration. Do you have all of the data you need to configure a fresh install of the system?
• Data Contract - What is the classification of the data stored in the system?
• Would we store sensitive data?
• Does the product carry / store / utilize / transport any VT-sensitive data ?
• Does the product own sensitive data ?
• What types of database access does your service require/allow?

Software

All

• Specific privileges required for installation and operation must be documented.
• All files installed or altered must be documented.
• Separation of privileged installation steps is preferred. (Think about the "run this script as root" part of an Oracle client install. That's a lot easier than looking at the whole install script.)
• We do not allow.
• Ask questions so we get a true answer.
• Does the vendor require remote access for problem resolution? If so, what level?
• The system administrator should be able to specify the install locations, e.g. /usr/local, /usr/local/depot, /opt, etc. (This often allows install by non-privileged user. I use opt_depot and do most installs as a non-root user).
• What privileges are required for day to day operation of the product ?

Client Components

• Evaluate the desk-top client from the point of view that the entire client machine and application can fall into the hands of an attacker.
  • What data are stored on the client machine in what format?
  • How is the connection to the client secured?
  • What components are required that are actually server?
  • The cliernt acts as a server - have server components (eg: file server, peer to peer local caching of data).

Server Components

 On what Operating System (OS) platforms is your service supported?

a. What OS version and release does your product support on those platforms?

b. What OS services are required for your product?

What ports does your product use?

OS version/patching

• How soon after the OS vendor releases a patch can it be applied to the system? (Systems with long lag times may require an external firewall.)
• How are upgrades and patches to your system tested and distributed?
• a. Can the customer apply these upgrades / patches?

Use of standard libraries

defined APIs

Hardware

Embedded OS

• How is configuration backed up if the device has to be replaced?
• How do you patch it?

Networking

• If the product can't operate on an open network, account for the cost of the firewall or special engineering required to protect it in the evaluation of the bid.
• All ports/protocols documented well enough to operate behind firewall?

AAA

Authorization:

How is access to your product controlled?

a. Can access permissions be assigned at multiple levels (group, departmantal, individual)?

b. Can document access be managed at the document section and field level?

Can administrative functions (assigning account permissions, for example) be delegated as opposed to being centrally performed?

How does your product authenticate and authorize users?

Does your system support multifactor authentication? For example, a token or smart card and PIN represent two factors: something you have and something you know.

Virginia Tech must comply with regulations such as FERPA and HIPPA. How does your product facilitate such compliance?

Authentication

External Auth

• Can the system use LDAP or CAS?
• Is there an authorization check in addition to authentication?

SSL Certificate auth

• Can an SSL certificate be used for authentication?
• Stored on security token?

Audit

Are you compliant (VISA, PCI) with audit standards for data center security? 

Can we see the components of the audit review - AICPA ?  Price-Waterhouse Audit Report?  

syslog

• Does logging allow determining the user who caused a particular event?
• Syslog facility and severity configurable?
• Is there an audit trail that documents administrative access and functions performed?
• a. Can these audit trails be copied or sent to a central server?
•  

debugging

Procurement

' i-Transact ' can process credit cards. Are we looking at a product that requires credit cards?

How do we raise awareness for software & liability that do not go though the' Request For Purchase' (RFP) process?  

Can we include minimums in 'Item For Bid' (IFB)?

Contract

Who is responsiible for the financial burden of notification of people if there is a compromise? This should be a part of contract discussions.

If our service experiences downtime, due to an unresolved security issue with your product, what type of financial compensation are we granted?

If the product is not in compliance with our procurement agreement. What do we do?  Return for refund?  Return product?  Fix product? 

Support

How are system or customer changes planned, reviewed, approved and documented?

How are systems monitored?

How does your company alert your customers to vulnerabilities and securtiy issues?

What are your technical support hours and response time for a reported incident?

What is the mean-time-to-fix an unknown problem?

Hosted Services

Disaster Recovery

What are disaster recovery capabilities?

What type of application and OS redundancy is designed into your system?

What is your guaranteed up time?

Access

What type of intrusion detection systems and firewalls are utilized on the servers that would host our systems?

Does your network or facility undergo penetration testing to ensure your systems cannot be hacked? Penetration testing involves your instructing someone to attempt to exploit vulnerabilities in your systems from outside your internal network.

a. If penetration testing is done, who performs it?

b. May we know the results?        

 Administrative   

                                                                                                                                                              
 What other services are installed on the server on which our service resides?
If other services reside on the same server as ours, how is our service configured to segregate it from the others?
What are your password rules for operating system administrators and users of our server?
How is our service isolated from internal company and test systems?
Do you perform vulnerability scanning  on your servers?
 Administrative  
How many people administer the server on which our system resides?
How many people have accounts on the server that would host our service?
a. How are those accounts provisioned & decommissioned?
b. Are default accounts disabled?
How is your facility physically secured?
How are patches and upgrades tested and applied?
a. When are patches applied?
b. How are emergency patches/upgrades handled?
   

Links

• The Fallacy of Trusted Client Software, By Bruce Schneier
• Snake Oil, By Bruce Schneier
• Snake Oil Warning Signs: Encryption Software to Avoid, By Matt Curtin

Leftover Bad Stuff

Here are some things that can happen but need to be turned in to something that can be evaluated above.

• Passwords obscured in executable or data files.
• Session hijacking