Background

Over recent years, science in Europe and worldwide has become increasingly reliant on high quality computing infrastructures enabling collaborative multi-disciplinary research. Grid computing technologies have been developed over the last decade to provide such a computing infrastructure.

Europe has played a leading role in this development, starting with the EU DataGrid Project (EDG) and related efforts under the fifth Framework Programme. After this proof of principle demonstrating the potential impact of Grid technologies on European science, a first large scale production Grid infrastructure was deployed by the Enabling Grids for E-SciencE project (EGEE), and its operation was further consolidated in its second phase (EGEE-II). The EGEE programme, which builds on national and thematic Grid efforts, as well as the pan-European network provided by GÉANT2 and the NRENs, aims to deliver Grid technology to many disparate disciplines and to users with widely varying levels of computing expertise located all over the globe. Since the beginning of EGEE-II, there has been a substantial expansion in the use of the infrastructure by a broad range of scientific applications. Scientists are clustered into Virtual Organisations (VOs) which gives them a framework to conduct collaborative research and gain access to shared computing and data resources. EGEE provides a variety of services to scientists, ranging from training and user support, through to the software infrastructure necessary to access the resources. The number of EGEE VOs supported now exceeds 250, and about 300,000 jobs/day are routinely executed worldwide in an infrastructure comprising more than 300 sites in over 50 countries. These applications deal with more than 5 PetaBytes of data made available on the EGEE infrastructure which sees routine data transfers of more than 1GB/sec. The number of scientists benefiting from EGEE has grown to over 10,000.

Life Sciences and High Energy Physics (HEP) now depend on the EGEE Grid infrastructure, which is an essential and crucial part of their large-scale data processing. Other disciplines, notably astrophysics, earth sciences, computational chemistry, and fusion to name but a few, are increasingly using it for production processing. Close relations have been established with a large number of collaborating projects extending the infrastructure and applications support to more disciplines. Through these collaborating projects and interoperability efforts with related non-European infrastructures, EGEE contributes to the establishment of a seamless worldwide computing infrastructure.

Continuous advancements of EGEE’s relations with business have contributed significantly to the growing uptake of Grid technologies in the commercial sector, with first prototype service providers appearing on the market and several sectors deploying enterprise Grid infrastructures.

Goals for EGEE-III

Given the success of the EGEE programme, it is now essential to build on its achievements and prepare the transition towards a sustainable infrastructure. This is proposed in the EGEE-III project, undertaken in close collaboration with National Grid Infrastructures (NGIs) and the European Grid Initiative Design Study project recently approved under FP7 (EGI_DS). The goal of the EGI_DS project is to devise a conceptual set-up of a new organisational model, based on NGIs such as the UK National Grid Service (NGS) or D-GRID in Germany, for a sustainable pan-European Grid infrastructure. EGEE-III will work closely with EGI_DS to transfer its unrivalled experience in operating large-scale international Grid infrastructures, ensuring the development of a viable model in EGI_DS. Based on the plans to be produced by EGI_DS, EGEE-III will, in its second year, start implementing the required structural changes to allow a seamless transition to the European Grid Initiative (EGI) model, while ensuring the continued provision of the Grid service. Figure 1 below illustrates the maturing status of the EGEE infrastructure since FP5, culminating in a sustainable Grid infrastructure in Europe.

A second major goal of the EGEE-III project is to maintain, enhance and simplify the use of the production quality computing infrastructure for an increasing range of researchers in diverse scientific fields, thus strengthening Europe’s leading position in high quality research. EGEE-III will continue to develop its reference open-source middleware distribution, gLite, that enables the Grid infrastructure. The medium-term goal is that commercial components can successively become an integral part of the infrastructure through the project’s proactive contributions to open standards and close collaboration with the IT industry. Further, by constructing the consortium on a country level through the implementation of National Grid Initiatives or Joint Research Units (a mechanism introduced by the EC in FP6 allowing partners to organise themselves in sub-consortia), EGEE-III will strengthen the role of national Grid infrastructures, thus exercising a structuring effect on the European Research Area (ERA).

EGEE-III plans to achieve these goals by a two-fold approach, ensuring a continuous service to an expanding user base and in parallel implementing structural changes for a sustainable Grid infrastructure through:

• provision of continuous Grid infrastructure operation with world-wide coverage for multi-disciplinary collaborative research, which can cope with the expected growth in resources and utilisation;
• The collaboration with strategic scientific disciplines as a primary source of innovation and interaction with user communities. In particular, the project will liaise closely with the European Strategy Forum on Research Infrastructure (ESFRI) projects defined in the ESFRI roadmap as part of its long-term goals;
• The provision of high-quality support to a wide range of VOs by providing services comprising training, dissemination of information, application porting and online user helpdesks;
• The encouragement and establishment of best security practices across all aspects of Grid computing, namely applications, middleware, deployment and operations to increase the level of trust and confidence in the Grid infrastructure;
• The provision of reliable and performant Grid middleware foundation services, as well as key, high-level middleware services, to users with integration and deployment facilities;
• Enhanced technology transfer to business communities;
• Ensuring the overall world-wide coherence of the developments by collaboration with other Grid projects, European and worldwide networking organisations, and standards organisations;
• The implementation of structural changes in concertation with the EGI_DS project, by collaborating with NGIs to facilitate the move towards sustainable Grid infrastructures in Europe;
• Encouraging the formation of National Grid Infrastructures (NGIs) through the use of Joint Research Unit (JRU) mechanism in countries where they are not yet initiated. These JRUs will represent nascent NGIs in over 20 European countries.

EGEE-III will take key steps in moving Grids to dependable and sustainable production infrastructure while providing a continuous service to its expanding user base. The goals presented will be achieved by a combination of Networking, Service and Joint Research activities. These activities will work in concert to maintain and drive the infrastructure operations, provide the necessary training, support and dissemination for existing and new user communities that use the infrastructure, and work in close collaboration to ensure the infrastructure is prepared for sustainability. A new user community (business or scientific) first makes contacts with EGEE-III through outreach events organised by the Networking Activities. Follow-up meetings by applications specialists lead to an experimental phase and a detailed deployment plan. The introduction of new applications and new user communities may lead to new requirements causing additions or modifications to be made to the Grid middleware via the Middleware Engineering Activity or collaborating projects. The Networking Activities then provide appropriate training to the community in question, so that it becomes an established user. Peer communication and dissemination events featuring established users then attract new communities. This “virtuous" cycle is depicted in the figure below.



EGEE-III Grid Services

The EGEE grid infrastructure consists of a set of middleware services deployed on a worldwide collection of computational resources. The EGEE project provides three services to users:

• Production Service: This is the largest grid infrastructure provided by EGEE. It runs the latest stable version of the gLite middleware. This is the preferred service for large-scale, production use of the grid.
• Preproduction Service: This service consists of a limited number of sites running a preview of the next release of the gLite software. This should be used to test existing applications against the new release, and to understand new gLite services.
• GILDA t-infrastructure: This is a grid which runs the entire gLite software stack in parallel to application porting support. Through other activities, the Production and Preproduction Services. It is used to demonstrate EGEE grid technology project also provides beginner and to support expert training courses.