Web Site Test Tools and Site Management Tools?
Load and Performance Test Tools
Tsung – Free open-source multi-protocol distributed load testing tool supported by Process-One. Can be used to stress HTTP, WebDAV, SOAP, PostgreSQL, MySQL, LDAP and Jabber/XMPP servers. SSL is also supported. OS monitoring (CPU, memory and network traffic) using SNMP, Munin or Erlang agents on remote servers. XML configuration system; several sessions can be used to simulate different type of users. Dynamic sessions can be described in XML. User think-times and the arrival rate can be randomized using a probability distribution. HTML reports can be generated during the load to view response times measurement, server CPU, etc. Developed in Erlang.
PerformanceXpert – Performance and load-testing solution available as a service over the Internet. Includes unlimited hardware and software already pre-configured and ready to run on-demand. Can realistically simulate thousands of virtual users and execute various load scenarios from multiple geographic locations (North America, Europe, and Asia). Includes web-based test management, archiving, repository, cloud-based monitoring, rich scripting language, and supports HTTP, HTTPS, web-services, XML, TCP, SQL, Login and more. Utilizes JMeter and Selenium. Lite and Pro versions.
LoadUI – Free open source cross-platform load testing tool from Eviwear/SmartBear Software. Using the soapUI Runner component (also from Eviwear/SmartBear), can leverage pre-existing functional soapUI TestCases and run them in loadUI. This integration enables support for HTTP(S), HTML, SOAP/WSDL and REST to AMF, JDBC, JMS and POX. Using loadUI Agents, can distribute loadUI TestCases to any number of Agents locally and remotely. Comprehensive analystics/reporting capabilities.
AppViewWeb – Cloud-based performance testing service from AppNeta that provides visibility into the network performance of web applications. Especially useful for QA test engineers conducting application pre-deployment testing on WAN networks. When testing web-based applications for CRM, VoIP, Video, Citrix, VMware and database management applications it’s helpful to understand the performance of these applications from the perspective out to remote site end users; AppViewWeb provides such insight.
Multi-Mechanize – Multi-Mechanize is an open source framework by Corey Goldberg for web performance and load testing. It allows you to run simultaneous python scripts to generate load (synthetic transactions) against a web site or web service. Results can be saved in CSV format along with an HTML report containing stats and graphs. Proficiency with Python, HTTP, and performance/load testing is recommended to use Multi-Mechanize successfully.
Load2Test – Performance/load/stress/high availability testing tool from Enteros Inc. Can capture real production workload for playback simulating thousands of users for top down testing or back-end testing at the component layer. Also available in the Amazon EC2, RackSpace or PlatForm Labs cloud environments. Integrated performance management and root cause analysis system automatically collects performance metrics across load test infrastructure and applies cross-component correlation to identify performance degradations and bottlenecks. Targeted platforms/OS’s: Load2Test Controller: Windows, Linux; Load2Test Test Nodes: Windows, Linux. Load2Test performance monitors and root cause analysis data collectors – OS: Windows, Linux, HP/UX, SUN Solaris, IBM AIX Databases: Oracle, DB2, SQL Server, Sybase, MySQL, PostgreSQL; App: .NET, JBoss, WebLogic, WebSphere, Oracle Application Server (OAS), GlassFish, Tomcat, JettyStorage: NetApp Filers, IBM DS8300
SiteBlaster – Web site load and stress testing tool; shareware. Can be used to rapidly submit requests to a site, or can pause a random amount of time between submissions, approximating user behavior. During testing the pages being tested will be displayed. Reports created on test completion. Designed to be very easy to use; intended for software developers and architects who want some early indication about performance characteristics of the web sites they create. Simulates MS IE web browsing functionality; a web page that is well behaved in IE should be well behaved in SiteBlaster. Best used to test those sites that use URL query strings to pass data to its web page(s). PDF user guide available. For Windows.
Load-Intelligence – Affordable load-testing ?Software as a Service? from Cloud-Intelligence. Software and unlimited hardware all included. JMeter users can execute their test-scripts in an unlimited, pre-configure, distributed environment. Neither setup nor installation are required. Immediate access to JMeter logs, reports, test script, CSV files and more.
LoadStorm – A web-based load testing tool/service as a distributed application that leverages the power of Amazon Web Services to scale on demand with processing power and bandwidth as needed. As the test loads increase to hundreds or thousands of virtual users, LoadStorm automatically adds machines from Amazon’s server farm to handle the processing. Tests can be built using the tool in such a way as to simulate a large number of different users with unique logins and different tasks.
BrowserMob – On-demand, self-service, low-cost, pay-as-you-go service from Neustar enables simulation of large volumes of real browsers hitting a website. Utilizes Amazon Web Services, Selenium. Uses real browsers for each virtual user so that traffic is realistic, AJAX & Flash support is automatic. Browser screen shots of errors included in reports.
Load Impact – Online load testing service from Gatorhole/loadimpact.com for load- and stress- testing of your website over the Internet; access to our distributed network of load generator nodes – server clusters with very fast connections to enable simulation of tens of thousands of users accessing your website concurrently. Free low level load tests for 1-50 simulated users; higher levels have monthly fees.
Pylot – Open source tool by Corey Goldberg for generating concurrent http loads. Define test cases in an XML file – specify requests – url, method, body/payload, etc – and verifications. Verification is by matching content to regular expressions and with HTTP status codes. HTTP and HTTPS (SSL) support. Monitor and execute test suites from GUI (wxPython), and adjust load, number of agents, request intervals, ramp-up time, test duration. Real-time stats and error reporting are displayed.
AppLoader – Load testing app from NRG Global for web and other applications accessible from a Windows desktop; generates load from the end user’s perspective. Protocol independent and supports a wide variety of enterprise class applications. Integrates with their Chroniker monitoring suite so results of load testing can be correlated with system behavior as load is increased. Runs from Win platforms.
fwptt – Open source tool by Bogdan Damian for load testing web applications. Capabilities include handling of Ajax. Generates tests in C#. For Windows platforms
JCrawler – An open-source stress-testing tool for web apps; includes crawling/exploratory features. User can give JCrawler a set of starting URLs and it will begin crawling from that point onwards, going through any URLs it can find on its way and generating load on the web application. Load parameters (hits/sec) are configurable via central XML file; fires up as many threads as needed to keep load constant; includes self-testing unit tests. Handles http redirects and cookies; platform independent.
Curl-Loader – Open-source tool written in ‘C’, simulating application load and behavior of tens of thousand HTTP/HTTPS and FTP/FTPS clients, each with its own source IP-address. In contrast to other tools curl-loader is using real C-written client protocol stacks, namely, HTTP and FTP stacks of libcurl and TLS/SSL of openssl. Activities of each virtual client are logged and collected statistics include information about: resolving, connection establishment, sending of requests, receiving responses, headers and data received/sent, errors from network, TLS/SSL and application (HTTP, FTP) level events and errors.
Gomez Web Load Testing – An on-demand load testing service from Gomez.com/Compuware. Utilizes Gomez?s Active Network providing on-demand active monitoring from 150+ enterprise-grade servers running in world-class, high-bandwidth data centers around the globe, and Gomez?s Active Last Mile which provides on-demand active monitoring from 150,000+ real, consumer-grade desktops in 168+ countries.
StressTester – Enterprise load and performance testing tool for web applications from Reflective Solutions Ltd. Advanced user journey modeling, scalable load, system resources monitors and results analysis. No scripting required. Suitable for any Web, JMS, IP or SQL Application. OS independent.
The Grinder – A Java-based load-testing framework freely available under a BSD-style open-source license. Orchestrate activities of a test script in many processes across many machines, using a graphical console application. Test scripts make use of client code embodied in Java plug-ins. Most users do not write plug-ins themselves, instead using one of the supplied plug-ins. Comes with a mature plug-in for testing HTTP services, as well as a tool which allows HTTP scripts to be automatically recorded.
Proxy Sniffer – Web load and stress testing tool from from Ingenieurb?ro David Fischer GmbH Capabilities include: HTTP/S Web Session Recorder that can be used with any web browser; recordings can then be used to automatically create optimized Java-based load test programs; automatic protection from “false positive” results by examining actual web page content; detailed Error Analysis using saved error snapshots; real-time statistics.
Testing Master – Load test tool from Novosoft, capabilities include IP spoofing, multiple simultaneous test cases and website testing features for sites with dynamic content and secure HTTPS pages.
JKool Online – Performance measurement and monitoring service from Nastel Inc. for web-based J2EE and SOA applications; start and stop live data monitoring whenever needed; drills down to JMS, JDBC, method calls, servlets and sessions with simple one-click option to view live session details; built-in support for jBoss, WebLogic and IBM WebSphere Application Server.
Funkload – Free web load testing, stress testing, and functional testing tool by Benoit Delbosc written in Python and distributed as free software under the GNU GPL. Emulates a web browser (single-threaded) using webunit; https support; produces detailed reports in ReST, HTML, or PDF.
Avalanche – Load-testing appliance from Spirent Communications, designed to stress-test security, network, and Web application infrastructures by generating large quantities of user and network traffic. Simulates as many as two million concurrently-connected users with unique IP addresses, emulates multiple Web browsers, supports Web Services testing Supports HTTP 1.0/1.1, SSL, FTP, RTSP/ RTP, MS Win Media, SMTP, POP3, DNS, Telnet, and Video on Demand over Multicast protocols.
Loadea – Stress testing tool runs on WinXP; free evaluation version for two virtual users. Capture module provides a development environment, utilizes C# scripting and XML based data. Control module defines, schedules, and deploys tests, defines number of virtual users, etc. Analysis module analyzes results and provides reporting capabilities.
LoadManager – Load, Stress, Stability and Performance testing tool from Alvicom. Runs on all platforms supported by Eclipse and Java.
QEngine Performance Tester – Automated testing tool from Zoho Corp. for performance testing (load and stress testing) of web applications and web services; J2EE, .NET, AJAX, PHP, Ruby on Rails, SOAP Web Services etc. Supports multiple browsers on Linux and Windows.
LoadComplete – Automated web load test tool from SmartBear Software.
QTest – Web load testing tool from Quotium Technologies SA. Capabilities include: cookies managed natively, making the script modelling phase shorter; HTML and XML parser, allowing display and retrieval of any element from a HTML page or an XML flux in test scripts; option of developing custom monitors using supplied APIs; more.
Test Perspective Load Test – Do-it-yourself load testing service from Keynote Systems for Web applications. Utilizes Keynote’s load-generating infrastructure on the Internet; conduct realistic outside-the-firewall load and stress tests to validate performance of entire Web application infrastructure.
SiteTester1 – Load test tool from Pilot Software Ltd. Allows definition of requests, jobs, procedures and tests, HTTP1.0/1.1 compatible requests, POST/GET methods, cookies, running in multi-threaded or single-threaded mode, generates various reports in HTML format, keeps and reads XML formatted files for test definitions and test logs. Requires JDK1.2 or higher.
httperf – Web server performance/benchmarking tool from HP Research Labs. Provides a flexible facility for generating various HTTP workloads and measuring server performance. Focus is not on implementing one particular benchmark but on providing a robust, high-performance, extensible tool. Available free as source code.
WAPT – Web load and stress testing tool from SoftLogica LLC. Handles dynamic content and HTTPS/SSL; easy to use; support for redirects and all types of proxies; clear reports and graphs.
http://www.microsoft.com/visualstudio/en-us/products/2010-editions/test-professional/overview Visual Studio Test Professional 2010 – A suite of testing tools for Web applications and services that are integrated into the Microsoft Visual Studio environment. These enable testers to author, execute, and manage tests and related work items all from within Visual Studio. Includes Lab Management capabilities.
OpenLoad – Affordable and completely web-based load testing tool from OpenDemand; knowledge of scripting languages not required – web-based recorder can capture and translate any user action from any website or web application. Generate up to 1000 simultaneous users with minimum hardware.
Apache JMeter – Java desktop application from the Apache Software Foundation designed to load test functional behavior and measure performance. Originally designed for testing Web Applications but has since expanded to other test functions; may be used to test performance both on static and dynamic resources (files, Servlets, Perl scripts, Java Objects, Data Bases and Queries, FTP Servers and more). Can be used to simulate a heavy load on a server, network or object to test its strength or to analyze overall performance under different load types; can make a graphical analysis of performance or test server/script/object behavior under heavy concurrent load.
TestMaker – Free open source utility maintained by PushToTest.com and Frank Cohen, for performance, scalability, and functional testing of Web application. Features test authoring of Web applications, Rich Internet Applications (RIA) using Ajax, Service Oriented Architecture, and Business Process Management environments. Integrates Selenium, soapUI, TestGen4Web, and HTMLUnit to make test development faster/easier. Repurposes tests from these tools into load and performance tests, functional tests, and business service monitors with no coding. Repurposes unit tests written in Java, Jython, JRuby, Groovy, and other dynamic scripting languages. Runs on any platform.
Siege – Open source stress/regression test and benchmark utility; supports basic authentication, cookies, HTTP and HTTPS protocols. Enables testing a web server with a configurable number of concurrent simulated users. Stress a single URL with a specified number of simulated users or stress multiple URL’s simultaneously. Reports total number of transactions, elapsed time, bytes transferred, response time, transaction rate, concurrency, and server response. Developed by Jeffrey Fulmer, modeled in part after Lincoln Stein’s torture.pl, but allows stressing many URLs simultaneously. Distributed under terms of the GPL; written in C; for UNIX and related platforms.
JBlitz – Load, performance and functional test tool from Clan Productions. Runs multiple concurrent virtual users.to simulate heavy load. Validates each response using plain text or regular expression searches, or by calling out to your own custom code. Full Java API. For testing and ‘bullet-proofing’ server side software – ASPs, JSPs, servlets, EJBs, Perl / PHP / C / C++ / CGI scripts etc.
WebServer Stress Tool – Web stress test tool from Paessler AG handles proxies, passwords, user agents, cookies, AAL.
Web Polygraph – Freely available benchmarking tool for caching proxies, origin server accelerators, L4/7 switches, and other Web intermediaries. Other features: for high-performance HTTP clients and servers, realistic traffic generation and content simulation, ready-to-use standard workloads, powerful domain-specific configuration language, and portable open-source implementation. C++ source available; binaries avail for Windows.
OpenSTA – ‘Open System Testing Architecture’ is a free, open source web load/stress testing application, licensed under the Gnu GPL. Utilizes a distributed software architecture based on CORBA. OpenSTA binaries available for Windows.
PureLoad – Java-based multi-platform performance testing and analysis tool from PureLoad Software. Includes ‘Comparer’ and ‘Recorder’ capabilities, dynamic input data, scenario editor/debugger, load generation for single or distributed sources.
ApacheBench – Perl API for Apache benchmarking and regression testing. Intended as foundation for a complete benchmarking and regression testing suite for transaction-based mod_perl sites. For stress-testing server while verifying correct HTTP responses. Based on the Apache 1.3.12 ab code. Available via CPAN as .tar.gz file.
Torture – Bare-bones Perl script by Lincoln Stein for testing web server speed and responsiveness and test stability and reliability of a particular Web server. Can send large amounts of random data to a server to measure speed and response time of servers, CGI scripts, etc.
WebSpray – Low-cost load testing tool from CAI Networks; includes link testing capabilities; can simulate up to 1,000 clients from a single IP address; also supports multiple IP addresses with or without aliases. For Windows.
eValid LoadTest – Web test tool from Software Research, Inc that uses a ‘Test Enabled Web Browser’ test engine that provides browser based 100% client side quality checking, dynamic testing, content validation, page performance tuning, and webserver loading and capacity analysis.
Load Testing by Web Performance – Load test tool emphasizing ease-of-use, from Web Performance Inc. Supports all browsers and web servers; records and allows viewing of exact bytes flowing between browser and server; no scripting required. Modem simulation allows each virtual user to be bandwidth limited. Can automatically handle variations in session-specific items such as cookies, usernames, passwords, IP addresses, and any other parameter to simulate multiple virtual users. For Windows, Linux, Solaris, most UNIX variants.
Optima Quality Studio – A collection of load testing, capture/playback, and related tools from Technovations for performance testing of web sites. Modules include WebCorder, Load Director, Report Generator, Batch, Manager, and others. WebSizer load testing module supports authentication, SSL, cookies, redirects. Recorded scripts can be modified manually. For Windows.
FORECAST – Load testing tool from Facilita Software for web, client-server, network, and database systems. Capabilities include proprietary, Java, or C++ scripting; windows browser or network recording/playback. Supports binary encoded data such as Adobe Flex/AMF, Serialised Java objects etc.SSL; supports NTLM, kerberos, proxies, authentication, redirects, certificates, cookies, caching, bandwidth limitation and page validation. Virtual user data can be parameterized. Works with a wide variety of platforms.
http-Load – Free load test application from ACME Labs to generate web server loads, from ACME Software. Handles HTTP and HTTPS; for Unix.
QALoad – Tool from Microfocus (formerly from Compuware) for load/stress testing of web, database, and character-based systems. Supports HTTP, SSL, SOAP, XML, Streaming Media. Works with a variety of databases, middleware, ERP.
IBM Rational Performance Tester – Performance testing tool from IBM/Rational; has optional extensions to Seibel applications and SAP Solutions. Supports Windows, Linux and z/OS as distributed controller agents; provides high-level and detailed views of tests.
SilkPerformer – Enterprise-class load-testing tool from Microfocus (formerly from Borland, formerly from Segue). Can simulate thousands of users working with multiple protocols and computing environments. Allows prediction of behavior of e-business environment before it is deployed, regardless of size and complexity.
Loadrunner – HP’s (formerly Mercury’s) load/stress testing tool for web and other applications; supports a wide variety of application environments, platforms, and databases. Large suite of network/app/server monitors to enable performance measurement of each tier/server/component and tracing of bottlenecks.
Return to top of web tools listing
Java Test Tools
Arquillian – Arquillian from JBoss enables you to test Java business logic in a remote or embedded container. Alternatively, it can deploy an archive to the container so the test can interact as a remote client. No longer does writing a test involve system administration tasks, custom scripts, copy-paste Maven configuration, full builds, test classpath mayhem, looking up resources manually or reliance on coarse-grained, black-box testing. The goal is to provide a simple test harness that abstracts away all container lifecycle and deployment from the test logic so developers can easily produce a broad range of integration tests for their enterprise Java applications. Arquillian can either execute a test case inside the container, in which case the test class is deployed by Arquillian along with the code under test, or hold back the test class so it can act as a remote client to the deployed code. All the developer has to do is write the test logic.
RTI – Java performance analysis tool from OC Systems Inc. helps quickly find and resolve complex performance problems. RTI delivers lightweight deep diagnostics for distributed Java applications in production and test environments. Monitor transaction performance end-to-end from client across network, web and application tiers ? deep dive to method level. RTI integrates with LoadRunner, JMeter and soapUI. Targeted to JBoss, Tomcat, Linux, Windows.
Yourkit Java Profiler – Java profiling tool from Yourkit; CPU and memory hot spots; memory leak detection; memory distribution reports;on-demand profiling; utilizes all of the advanced Java 5/6/7 profiling features; runs on most platforms.
VisualVM – A free visual tool, originally from Sun, to monitor and troubleshoot Java applications. Runs on Sun JDK 6, but is able to monitor applications running on JDK 1.4 and higher. Utilizes various available technologies like jvmstat, JMX, the Serviceability Agent (SA), and the Attach API to get data and uses minimal overhead on monitored applications. Capabilities include: automatically detects and lists locally and remotely running Java applications; monitor application performance and memory consumption; profile application performance or analyze memory allocation; is able to save application configuration and runtime environment together with all taken thread dumps, heap dumps and profiler snaphots into a single application snapshot which can be later processed offline.
LAPSE – Lightweight Analysis for Program Security in Eclipse. Designed to help with the task of auditing Java J2EE applications for common types of security vulnerabilities found in Web apps. Targets parameter manipulation, header manipulation, cookie poisoning, command-line parameters, SQL injections, cross-site scripting, HTTP splitting, path traversal vulnerabilities Developed by Benjamin Livshits as part of the Griffin Software Security Project. Not intended as a comprehensive solution for Web application security, but rather as an aid in the code review process.
Checkmarx – Code security analysis tool from Checkmarx; capabilitites include hundreds of out of the box security checks, checks for architectural flaws, coding practice enforcement. Runs on Windows against Java or any other programming language code.
Window Licker – An open source framework for the test-driven development of Java systems through the GUI. Provides a high-level API for controlling and making assertions about graphical user interfaces: for Swing and Dynamic HTML (aka “AJAX”) including GWT. Deals with the asynchronous nature of GUI and AJAX programming so the tests don’t have to; Controls the GUI by sending native mouse and keyboard events; Handles different keyboard layouts; Produces high quality error messages to help you easily diagnose test failures; Easily extensible to cope with new user interface components
Cobertura – Free Java tool to identify which parts of a Java program are lacking test coverage and calculate % coverage; based on jcoverage. Instruments already-compiled Java bytecode; execute from ant or from the command line; generate reports in HTML or XML; shows % of lines and branches covered for each class, each package, and for the overall project. Shows McCabe cyclomatic code complexity of each class, and average cyclomatic code complexity for each package, and for the overall product. Can sort HTML results by class name, percent of lines covered, percent of branches covered, etc. and sort in ascending or descending order.
JProfiler – Java profiling tool from ej-Technologies GmbH. Check for performance bottlenecks, memory leaks and threading issues.
Parallel-junit – Open source small library extensions for JUnit. Extends the junit.framework.TestSuite class by running tests in parallel, allowing more efficient test execution. Because TestResult and TestListener aren’t designed to run tests in parallel, this implementation coordinates the worker threads and reorder event callbacks so that the TestResult object receives them in an orderly manner. In addition, output to System.out and System.err are also serialized to avoid screen clutter.
EMMA – Open-source toolkit, written in pure Java, for measuring and reporting Java code coverage. Targets support for large-scale enterprise software development while keeping individual developer’s work fast and iterative. Can instrument classes for coverage either offline or on the fly (using an instrumenting application classloader); supported coverage types: class, method, line, basic block; can detect when a single source code line is covered only partially; coverage stats are aggregated at method, class, package, and “all classes” levels. Reports support drill-down, to user-controlled detail depth; HTML reports support source code linking. Does not require access to the source code; can instrument individual .class files or entire .jars (in place, if desired). Runtime overhead of added instrumentation is small (5-20%); memory overhead is a few hundred bytes per Java class.
PMD – Open source static analyzer scans java source for problems. Capabilities include scanning for: Empty try/catch/finally/switch statements; Dead code – unused local variables, parameters and private methods; Suboptimal code – wasteful string/stringBuffer usage; Overcomplicated expressions – unnecessary if statements, for loops that could be while loops; Duplicate code – copied/pasted code – could indicate copied/pasted bugs.
Hammurapi – Code review tool for Java (and other languages with latest version). Utilizes a rules engine to infer violations in source code. Doesn’t fail on source files with errors, or if some inspectors throw exceptions. Parts of tool can be independently extended or replaced. Can review sources in multiple programming languages, perform cross-language inspections, and generate a consolidated report. Eclipse plugin.
TestNG – A testing framework inspired from JUnit and NUnit and developed by Cedric Beust; supports JDK 5 Annotations, data-driven testing (with @DataProvider), parameters, distribution of tests on slave machines, plug-ins (Eclipse, IDEA, Maven, etc); embeds BeanShell for further flexibility; default JDK functions for runtime and logging (no dependencies); can run tests in arbitrarily big thread pools with various policies available.
Concordian – An open source testing framework for Java developed by David Peterson. Utilizes requirements in plain English using paragraphs, tables and proper punctuation in HTML. Developers instrument the concrete examples in each specification with commands (e.g. “set”, “execute”, “assertEquals”) that allow test scenarios to be checked against the system to be tested. The instrumentation is invisible to a browser, but is processed by a Java fixture class that accompanies the specification. The fixture is also a JUnit test case. Results are exported with the usual green and red indicating successes and failures. Site includes info re similarities and diffs from Fitnesse.
DBUnit – Open source JUnit extension (also usable with Ant) targeted for database-driven projects that, among other things, puts a database into a known state between test runs. Enables avoidance of problems that can occur when one test case corrupts the database and causes subsequent tests to fail or exacerbate the damage. Has the ability to export and import database data to and from XML datasets. Can work with very large datasets when used in streaming mode, and can help verify that database data matches expected sets of values.
StrutsTestCase – Open source Unit extension of the standard JUnit TestCase class that provides facilities for testing code based on the Struts framework, including validation methods. Provides both a Mock Object approach and a Cactus approach to actually run the Struts ActionServlet, allowing testing of Struts code with or without a running servlet engine. Uses the ActionServlet controller to test code, enabling testing of the implementation of Action objects, as well as mappings, form beans, and forwards declarations.
DDSteps – A JUnit extension for building data driven test cases. Enables user to parameterize test cases, and run them more than once using different data. Uses external test data in Excel which is injected into test cases using standard JavaBeans properties. Test cases run once for each row of data, so adding new tests is just a matter of adding a row of data in Excel.
StrutsTestCase for JUnit – Open source extension of the standard JUnit TestCase class that provides facilities for testing code based on the Struts framework. Provides both a Mock Object approach and a Cactus approach to actually run the Struts ActionServlet, allowing testing Struts code with or without a running servlet engine. Because it uses the ActionServlet controller to test code, can test not only the implementation of Action objects, but also mappings, form beans, and forwards declarations. Since it already provides validation methods, it’s quick and easy to write unit test cases.
JavaNCSS – A free Source Measurement Suite for Java by Clemens Lee. A simple command line utility which collects various source code metrics for Java. The metrics are collected globally, for each class and/or for each function.
Open Source Profilers for Java – Listing of about 25 open source code profilers for Java from 2006 from the Manageability.org web site.
SofCheck Inspector – Tool from SofCheck Inc. for analysis of Java for logic flaws and vulnerabilities. Explores all possible paths in byte code and detects flaws and vulnerabilities in areas such as: array index out of bounds, buffer overflows, race conditions, null pointer dereference, dead code, etc. Provides 100% path coverage and can report on values required for 100% unit test coverage. Patented precondition, postcondition and presumption reporting can help detect Malware code insertion.
CodePro – Suite of Java tools from Instantiations Inc. CodePro AnalytixX is an Eclipse-based Java software testing tool and includes features like code audit, metrics, automated unit tests, and more. CodePro Profiler, an Eclipse-based Java profiling tool enables inspection of a running application for performance bottlenecks, detect memory leaks and solve thread concurrency problems. EclipsePro Test automatically generates JUnit tests and includes an editor and analysis tool, provides test cases/results in tabular layout; mouse over failing case and Editor shows the failure message. WindowTester Pro for Swing or SWT UI’s enables recording of GUI tests; watches actions and generates test cases automatically; customize the generated Java tests as needed. Provides a rich GUI Test Library, hiding complexities and threading issues of GUI test execution; test cases are based on the JUnit standard
Klocwork – Static analysis technology for Java, C, C++, C#; analyzes defects & security vulnerabilities, architecture & header file anomalies, metrics. Developers can run Klocwork in Eclipse or various other IDE’s. Users can select scope of reporting as needed by selecting software component, defect type, and defect state/status.
Coverity Prevent – Tool from Coverity Inc. for analysis of Java source code for security issues. Explores all possible paths in source code and detects security vulnerabilities and defects in multiple areas: memory leaks, memory corruption, and illegal pointer accesses, buffer overruns, format string errors and SQL injections vulnerabilities, multi-threaded programming concurrency errors, etc.
GUIDancer – Eclipse-based tool from Bredex GmbH for automated testing of Java/Swing GUI’s, Tests are specified, not programmed – no code or script is produced. Test specification is initially separate from the AUT, allowing test creation before the software is fully functional or available. Specification occurs interactively; components and actions are selected from menus, or by working with the AUT in an advanced “observation mode”. Test results and errors viewable in a results view, can be saved as html or xml file.
CMTJava – Complexity measurement tool from Verifysoft GmbH. Includes McCabe cyclomatic complexity, lines-of-code metrics, Halstead metrics, maintainability index.
JavaCov – A J2SE/J2EE Coverage testing tool from Alvicom; specializes in testing to MC/DC (Modified Condition/Decision Coverage) depth. Capabilities include: Eclipse plugin; report generation into HTML and XML; Apache Ant integration and support for test automation.
Jameleon – Open source automated testing harness for acceptance-level and integration testing, written in Java. Separates applications into features and allows those features to be tied together independently, in XML, creating self-documenting automated test cases. These test-cases can then be data-driven and executed against different environments. Easily extensible via plug-ins; includes support for web applications and database testing.
Agitator – Automated java unit testing tool from Agitar Software. Creates instances of classes being exercised, calling each method with selected, dynamically created sets of input data, and analyzing results. Stores all information in XML files; works with Eclipse and a variety of IDEs. Also available are: automated JUnit generation, code-rule enforcement, and more.
PMD – Open source tool scans Java code for potential bugs, dead code, duplicate code, etc. – works with a variety of configurable and modifiable rulesets. Integrates with a wide variety of IDE’s.
JLint – Open source static analysis tool will check Java code and find bugs, inconsistencies and synchronization problems by doing data flow analysis and building the lock graph.
Lint4j – A static Java source and byte code analyzer that detects locking and threading issues, performance and scalability problems, and checks complex contracts such as Java serialization by performing type, data flow, and lock graph analysis. Eclipse, Ant and Maven plugins available.
FindBugs – Open source static analysis tool to inspect Java bytecode for occurrences of bug patterns, such as difficult language features, misunderstood API methods, misunderstood invariants when code is modified during maintenance, garden variety mistakes such as typos, use of the wrong boolean, etc. Can report false warnings, generally less than 50%.
CheckStyle – Open source tool for checking code layout issues, class design problems, duplicate code, bug patterns, and much more.
Java Development Tools – Java coverage, metrics, profiler, and clone detection tools from Semantic Designs.
AppPerfect Test Studio – Suite of testing, tuning, and monitoring products for java development from AppPerfect Corp. Includes: Unit Tester, Code Analyzer, Java/J2EE Profiler and other modules.
Cactus – A simple open-source test framework for unit testing server-side java code (Servlets, EJBs, Tag Libs, Filters, etc.). Intent is to allow fine-grained continuous testing of all files making up an application: source code but also meta-data files (such as deployment descriptors, etc) through an in-container approach. It uses JUnit and extends it. Typically use within your IDE, or from the command line, using Ant. From Apache Software Foundation.
JUnitPerf – Allows performance testing to be dynamically added to existing JUnit tests. Enables quick composition of a performance test suite, which can then be run automatically and independent of other JUnit tests. Intended for use where there are performance/scalability requirements that need re-checking while refactoring code. By Mike Clark/Clarkware Consulting, licensed under the BSD License.
Abbot Java GUI Test Framework – Testing framework by Timothy Wall provides automated event generation and validation of Java GUI components, improving upon the very basic functions provided by the java.awt.Robot class. (Abbot = “A Better ‘Bot’). The framework may be invoked directly from Java code or accessed without programming through the use of scripts via ‘Costello’, a script editor/recorder. Suitable for use both by developers for unit tests and QA for functional testing. Free – available under the GNU Lesser General Public License
JUnit – Framework to write repeatable java unit tests – a regression testing framework written by Erich Gamma and Kent Beck. For use by developers implementing unit tests in Java. Free Open Source Software released under the IBM Public License and hosted on SourceForge. Site includes a large collection of extensions and documentation.
jfcUnit – Framework for developing automated testing of Java Swing-based applications at the UI layer (as opposed to testing at lower layers, for which JUnit may be sufficient). Provides recording and playback capabilities. Also available as plugins for JBuilder and Eclipse. Free Open Source Software from SourceForge site.
JBench – Freeware Java benchmarking framework to compare algorithms, virtual machines, etc. for speed. Available as binary distribution (including documentation), source distribution, or jar file.
Clover – Code coverage tool for Java from Atlassian. Fully integrated plugin for Eclipse, IntelliJ IDEA and projects using Apache ANT and Maven. View coverage data in XML, HTML, PDF, or via a Swing GUI. Tracks cyclomatic complexity. TestOptimization automatically prioritises just the tests needed to cover the particular changes made.
Structure101 – Java source code visualization tool from Headway Software. Lets user understand, measure, and control architecture, design, composition, and dependencies of code base. Analyzes byte code and shows all dependencies, at all levels and between all levels; method, class, package, application. Measures code complexity using a measurement framework called XS. For Windows, Linux and Mac OS X.
Java Tool Suite from Man Machine Systems – Includes JStyle, a Java source analyzer to generate code comments and metrics such as inheritance depth, Cyclomatic Number, Halstead Measures, etc; JPretty reformats Java code according to specified options; JCover test coverage analyzer; JVerify Java class/API testing tool uses an invasive testing model allowing access to internals of Java objects from within a test script and utilizes a proprietary OO scripting language; JMSAssert, a tool and technique for writing reliable software; JEvolve, an intelligent Java code evolution analyzer that automatically analyzes multiple versions of a Java program and shows how various classes have evolved across versions; can ‘reason’ about selective need for regression testing Java classes; JBrowser class browser; JSynTest, a syntax testing tool that automatically builds a Java-based test data generator.
JProbe Suite – Collection of Java debugging tools from Quest Software; includes JProbe Profiler and JProbe Memory Debugger for finding performance bottlenecks and memory leaks, LProbe Coverage code coverage tool, and JProbe Threadalyzer for finding deadlocks, stalls, and race conditions. JProfiler freeware version available.
Krakatau Professional for Java – Software metrics tool from Power Software includes more than 70 OO, procedural, complexity, and size metrics related to reusability, maintainability, testability, and clarity. Includes Cyclomatic Complexity, Enhanced Cyclomatic Complexity, Halstead Software Science metrics, LOC metrics and MOOD metrics. Has online advisor for quality improvement.
Jtest – ParaSoft’s Jtest is an integrated, automatic unit testing and standards compliance tool for Java. It automatically generates and executes JUnit tests and checks whether code follows 400 coding standards and can automatically correct for many.
DevPartner Java Edition – Debugging/productivity tool from Microfocus (formerly from Compuware, formerly from NuMega) to detect and diagnose Java bugs and memory and performance problems; thread and event analysis, coverage analysis. Integrates with several Java IDE’s.
TCAT for Java – Part of Software Research’s TestWorks suite of test tools; code coverage analyzer and code analysis for Java; written in Java.
Open Source code analyzers listing – A listing of open source Java code analysis tools written in Java.
Open Source code coverage tools listing – A listing of open source Java code coverage tools written in Java.
Open Source Java test tools listing – A listing of open source tools and frameworks for Java testing, written in Java.
Open Source web test tools listing – A listing of open source web test tools and frameworks written in Java.
(Note: some other tools in these listings also handle testing, management, or load testing of java applets, servlets, and applications, or are planning to add such capabilities. Check listed web sites for current information.)
Link Checking Tools
LinkTiger – Hosted link checker; free and $pro versions. Capabilities include e-mail alerts, dashboard, reporting; canned reports or create rich custom reports. Scans PDF, CSS, Flash and MS Office files, flash-animation.
HiSoftware Link Validation Utility – Link validation tool; available as part of the AccVerify Product Line from HiSoftware Inc.
ChangeAgent – Link checking and repair tool from Expandable Language. Identifies orphan files and broken links when browsing files; employs a simple, familiar interface for managing files; previews files when fixing broken links and before orphan removal; updates links to moved and renamed files; fixes broken links with an easy, 3-click process; provides multiple-level undo/redo for all operations; replaces links but does not reformat or restructure HTML code. For Windows.
Link Checker Pro – Link check tool from KyoSoft; can also produce a graphical site map of entire web site. Handles HTTP, HTTPS, and FTP protocols; several report formats available. For Windows platforms.
Web Link Validator – Link checker from REL Software checks links for accuracy and availability, finds broken links or paths and links with syntactic errors. Export to text, HTML, CSV, RTF, Excel. Freeware ‘REL Link Checker Lite’ version available for small sites. For Windows.
Site Audit – Low-cost on-the-web link-checking service from Blossom Software.
Xenu’s Link Sleuth – Freeware link checker by Tilman Hausherr; supports SSL websites; partial testing of ftp and gopher sites; detects and reports redirected URL; Site Map; for Windows.
Linkalarm – Low cost on-the-web link checker from Link Alarm Inc.; free trial period available. Automatically-scheduled reporting by e-mail.
Alert Linkrunner – Link check tool from Viable Software Alternatives; evaluation version available. For Windows.
InfoLink – Link checker program from BiggByte Software; can be automatically scheduled; includes FTP link checking; multiple page list and site list capabilities; customizable reports; changed-link checking; results can be exported to database. For Windows. Discontinued, but old versions still available as freeware.
LinkScan – Electronic Software Publishing Co.’s link checker/site mapping tool; capabilities include automated retesting of problem links, randomized order checking; can check for bad links due to specified problems such as server-not-found, unauthorized-access, doc-not-found, relocations, timeouts. Includes capabilities for central management of large multiple intranet/internet sites. Results stored in database, allowing for customizable queries and reports. Validates hyperlinks for all major protocols; HTML syntax error checking. For all UNIX flavors, Windows, Mac.
CyberSpyder Link Test – Shareware link checker by Aman Software; capabilities include specified URL exclusions, ID/Password entries, test resumption at interruption point, page size analysis, ‘what’s new’ reporting. For Windows.
Total Validator – Java-based desktop tool validates HTML / XHTML / WCAG / Section 508 / Links / Spelling; free and pro versions. HTML validation against the W3C Specifications; accessibility validation against the WCAG (1.0 and 2.0) and US Section 508 standard; checks each page for broken links; spell checks (English, French, Italian, Spanish, German), 1-click validation using the Firefox/SeaMonkey extension, choice of browser identification to use when checking websites. Works on a wide varity of operating systems.
RealValidator – Shareware HTML validator based on SGML parser by Liam Quinn. Unicode-enabled, supports documents in virtually any language; supports XHTML 1.0, HTML 4.01, HTML 4.0, HTML 3.2, HTML 3.0, and HTML 2.0 ; extensible – add proprietary HTML DTDs or change the existing ones; fetches external DTDs by HTTP and caches them for faster validation; HTML 3.2 and HTML 4.0 references included as HTML Help. For Windows.
HTML Validator – Firefox add-on, open source by Marc Gueury. The validation is done on your local machine inside Firefox and Mozilla. Error count of an HTML page is seen as an icon in the status bar when browsing. Can validate the HTML sent by the server or the HTML in the memory (after Ajax execution). Error details available when viewing the HTML source of the page. Based on Tidy and OpenSP (SGML Parser). Available in 17 languages and for Windows and other platforms.
CSE 3310 HTML Validator – HTML syntax checker for Windows from AI Internet Solutions. Supports wide variety of standards; accessibility (508) checking; uppercase/lowercase converter. Free ‘lite’ version. For Windows.
(Note: Many of the products listed in the Web Site Management Tools section include HTML validation capabilities.)
Free On-the-Web HTML Validators and Link Checkers
Site Check – Type in one URL and automatically run HTML and stylesheet validators, accessibility assessment, link check, load time check, and more. Organizes access to a collection of free online web test tools. Site of UITest.com/Jens Meiert. Also lists a wide variety of free online web analysis/development/test tools.
CynthiaSays Portal – Free web content accessibility/508 validation tool; checks one page at a time. Designed to identify errors in content related to Section 508 standards and/or the WCAG guidelines. The HiSoftware CynthiaSays portal is a joint Education and Outreach project of HiSoftware, ICDRI, and the Internet Society Disability and Special Needs Chapter.
Link Valet – Online link checker, includes capability for hilighting links modified since a specified date.
WDG HTML Validator – Web Design Group’s validator – latest HTML version support, flexible input methods, user-friendly error messages.
Web Page ‘Purifier’ – Free on-the-web HTML checker by DJ Delorie allows viewing a page ‘purified’ to HTML 2.0, HTML 3.2, HTML 4.0, or WebTV 1.1. standards.
W3C HTML Validation Service – HTML validation site run by the WWW Consortium (the folks who set web standards); handles one URL at a time; Can choose from among 30 character encoding types, and multiple HTML and XHTML document types/versions.
W3C CSS Validation Service – CSS validation site run by the WWW Consortium (the folks who set web standards); handles one URI at a time; or upload file or validate by direct input.
W3C Link Checker – Link checking service run by the WWW Consortium (the folks who set web standards); configurable. Handles one URL at a time. PERL source also available for download.
Weblint Gateway – Site with online HTML validator; somewhat configurable. Site provided by San Francisco State University.
Web Page Backward Compatibility Viewer – On-the-web HTML checker by DJ Delorie; will serve a web page to you with various selectable tags switched on or off; very large selection of browser types; to check how various browsers or versions might see a page.
PERL and C Programs for Validating and Checking
W3C Link Checker – Link checker PERL source code, via the WWW Consortium (the folks who set web standards); configurable. Handles one URL at a time. See ‘Download’ link.
HTML TIDY – Free utility available from SourceForget.net; originally by Dave Raggett. For automatic fixing of HTML errors, formatting disorganized editing, and finding problem HTML areas. Available as source code or binaries.
Big Brother – Freeware command-line link checker for Unix, Windows, by Francois Pottier. Available as source code; binary available for Linux.
LinkLint – Open source Perl program checks local/remote HTML links. Includes cross referenced and hyperlinked output reports, ability to check password-protected areas, support for all standard server-side image maps, reports of orphan files and files with mismatching case, reports URLs changed since last checked, support of proxy servers for remote URL checking. Distributed under Gnu General Public License. Has not been updated in recent years.
MOMspider – Multi-Owner Maintenance Spider; link checker. PERL script for a web spider for web site maintenance; for UNIX and PERL. Utilizes the HTTP ‘HEAD’ request instead of the ‘GET’ request so that it does not require retrieval of the entire html page. This site contains an interesting discussion on the use of META tags. Not updated in recent years.
HTMLchek for awk or perl – Old but still useful HTML 2.0 or 3.0 validator programs for AWK or PERL by H. Churchyard; site has much documentation and related info. Not updated in recent years.
Cross-browser testing tools/services
Spoon – App emulation service that runs cloud-based applications from any windows desktop with no installs; requires installation of Spoon browser plugin. Provides free versions of Firefox, Chrome, Opera and Safari with multiple versions of each. IE is not included. Paid $ Spoon Server and Spoon Studio also available.
BrowserBox – Appliance for cross browser testing by Jacob Rask, a virtual machine that runs in VirtualBox, VMWare, or from live boot CD or USB memory stick. Has multiple browsers and versions, including some mobile browsers, but does not include IE8 or 9. Based on Linux distribution openSUSE, but don’t need to have Linux installed or have Linux experience.
Utilu – Free utilities from Utilu that contain collections of standalone versions of IE or Firefox browsers; multiple versions can be used at the same time. Utilu Mozilla Firefox Collection contains more then ten versions of Firefox (English versions only); also includes the Firebug and Web Developer add-ons and Flash player; configurable to install only specified desired version. Supports upgrading – no need to uninstall a previous version before installing a newer version. Utilu IE Collection contains more than ten versions of Internet Explorer (English versions only); configurable to install only specified desired version; original version number is shown correcty in the User Agent string; version number also shown in window title; includes the IE Developer Toolbar.
Adobe Browser Lab – Cross-browser testing online service from Adobe; free until 2012. Preview web pages across multiple browsers and operating systems, navigate links, and use diagnostic tools to optimize websites efficiently. Also supports Firebug extension for Firefox.
Browsercam – Online cross-browser testing service from Compuware/Gomez. Screen capture service lets you submit single or multiple URL’s, choose the browsers and operating systems you want to see, and screen captures of your webpage are loaded in the different browsers and operating systems you selected.
Browsershots – Free online cross browser/OS comparison testing. Choose browser OS, browser, and versions of interest and submit URL and site responds with a collection of screen shots.
Multi-Browser Viewer – Cross browser testing solution from TWD Solutions Pte Ltd. includes a wide variety of standalone virtualized browsers, multiple standalone mobile browsers or simulators, wide variety of screen capture image browsers; Screenshot comparison function (?Onion Skin? or ?Side-by-Side?); available in multiple languages.
BrowserSeal – Multi browser website screenshot tool – capture an image of web site under multiple browsers Supports multiple versions of IE, Firefox, Google Chrome, Opera, Safari. Optimized for speed. Comes with standalone versions of all major browsers to verify site layout issues and troubleshoot functionality issues. The Automated edition adds the ability to automatically capture multiple URLs without user intervention, and command-line control enabling use in automation scripts. For Windows platforms.
IE Netrenderer – Free site allows you to check how a website is rendered by Internet Explorer 9, 8, 7, 6 or 5.5, as seen from a high speed datacenter located in Germany. Just type in a URL. Able to process a large number of capturing jobs in parallel and in realtime, making for fast service. From Geotek IT.
Litmus – Cross-browser testing service from Salted Services Inc.; oriented to html email testing; but also does ‘landing page’ cross browser testing for 8 different web browsers.
Web Functional/Regression Test Tools
Fabasoft app.test – Web test tool from Fabasoft Distribution GmbH; tests are based on patterns to reduce the complexity of HTML and Java applications – no XPath or CSS expressions are stored in the test but meaningful and understandable statements. Patterns for web sites can be designed by using the point-and-click editor as an Eclipse Plugin. Tests can be recorded by using the point-and-click recorder in all supported web browsers (IE, Firefox, Safari) and immediately be replayed in all other browsers without modification. Various Ant-Tasks provide seamless integration into Continuous Integration systems like Hudson. Generates various reports as HTML or PDF; errors documented with screenshots and various dumps. Supports Win, Linux, Mac OS-X; free and premium $ versions available.
Janova – A web-based, automated web testing tool that runs functional tests securely in the cloud. Users configure Janova using project structures of Features (test scripts), Pages and Flows to access their web-based application and define the requirements of the site in English. Features describe how the application is supposed to work; once a feature has been created, a user defines what verification elements are supposed to be on the web page. Includes detailed test results reports.
Twist – Integrated Development Environment (IDE) from Thoughtworks Studios for agile functional testing of web apps and Java Swing apps. Uses an enhanced version of Selenium (”TwistSelenium”) to drive web app testing, which has many fixes on top of Selenium, and adds functionality such as a recorder. Capabilities include: Express requirements directly as test specifications in English (or any UTF-8 supported language) using your domain language; map test specifications to the underlying code and navigate between them with IDE-support; out-of-the-box support for Test Driven Development. Java-based – runs on Windows, Linux and Mac OS X.
QF-Test – QF-Test/web from Quality First Software is a cross-platform software tool for cross-browser test automation of web-based applications (HTML, AJAX, ExtJS, GWT, RAP, Qooxdoo, RichFaces) and Java apps. Includes small-scale test management capabilities, capture/replay mechanism, intuitive user interface and extensive documentation, reliable component recognition and can handle complex and custom GUI objects, integrated test debugger and customizable reporting.
Fabasoft app.test – Web test tool from Fabasoft Distribution GmbH; includes recorder, multi-browser support, dialog and script error handler, errors are documented with a screenshot and various dumps, various reports are generated and provided as HTML or PDF; more. Available as free and $ versions.
Cloud Testing Service – Web testing utilizing cloud capabilities, from Cloud Testing Limited. Involves first recording web functionality via browser and Selenium IDE, uploading scripts to the Cloud Testing website; then scripts run using real browsers on real operating systems in the testing cloud; results are available as screenshots, HTML & component diagnostics. Test can be re-run whenever needed or as scheduled. Cross Browser feature enables test runs in multiple browsers (IE, Firefox, Safari, Opera, Google Chrome) and comparing appearance/results side-by-side. Cloud load testing and site monitoring services also available.
SlimDog – A simple script based web application testing tool. It is based on httpunit. The tool offers a wide range of commands to work with forms, check the content of tables and navigation between HTML pages. Rather than writing long JUnit testcases or crucial XML files the users can write simple text scripts. Each line of the script file will contain one command whic
Log in to answer.
Copyright © dBuggr.com - All Rights Reserved.