Raymond G. Kammer
National Institute of Standards and Technology
House Committee on Science
Subcommittee on Technology
A review of the FY 1999 Administration Request for the Technology Administration and the National Institute of Standards and Technology
February 26, 1998
Chairwoman Morella and other Members of this Subcommittee, it is a real honor to testify before you today. I have appeared before this Subcommittee on a number of occasions over the past several years, but this is the first time that I come before you as Director of the National Institute of Standards and Technology. That is something that I am very proud of because I consider NIST to be the finest institution in the federal government, and I truly believe that I have the best job in the government.
These are times of rapid change, and there is much that we must do to ensure that NIST continues as an institution of excellence that fulfills its mission of working with industry to develop and apply technology, measurements and standards. I see five challenges for NIST as we approach the agency's centennial in 2001 -- its first century of service for America. They are:
- Ensuring world leadership by NIST's Measurement and Standards Laboratories,
- Ensuring that measurement capabilities and standards are in place to support full U.S. participation in global markets,
- Building greater consensus on the Advanced Technology Program's value,
- Expanding access to Manufacturing Extension Partnership services for more small and medium-sized companies and continuing federal support for MEP centers after the sixth year, and
- Promoting performance excellence in healthcare and education, particularly among non- profit organizations, through the Baldrige National Quality Program.
The President's fiscal year 1999 budget request for NIST of $715 million reflects our approach to those challenges. Each of these areas is linked closely with the Commerce Department's and NIST's strategic and performance plans, and we have worked hard on meaningful evaluation metrics to chart our progress in meeting these challenges. Our work is results-driven, and I will have more to say about our metrics later.
Measurement and Standards Laboratories
Our first challenge is world leadership for NIST's Measurement and Standards Laboratories. The United States has the largest and most vibrant economy in the world, and to maintain this position we must have the best measurements and standards in the world to support U.S. commerce and competitiveness. There has never been a time in which measuring accurately has been more important to the health of the U.S. economy. This is an age in which semiconductor devices shrink to half their previous size every three years and measurement advances depend on microscopes that "see" individual atoms. The longevity and reliability of engines depend upon manufacturing tolerances of micrometers--about the width of a single bacterium. Compact disc players already incorporate lenses to focus laser lights to tolerances below a tenth of a micrometer. Our global communication systems increasingly rely on accurate clocks synchronized within a millionth of a second.
NIST conducts research on fundamental measurement techniques and technologies that individual companies typically have neither the technical ability nor the resources to conduct on their own. While each company needs these new measurement technologies and standards, once developed, the benefits extend to all. A company paying for such research could not expect to recoup its costs -- and it would be paying for research that would benefit not only itself and its customers, but also its competitors.
The NIST Measurement and Standards Laboratories stand atop the nation's "measurement pyramid." They maintain, for example, approximately 1,300 different Standard Reference Materials -- the equivalents of certified "rulers" that companies, government agencies, and others use to check the accuracy of their most exacting measurements. These materials broadly support the private-sector market in secondary reference materials used by businesses, laboratories and other organizations to certify their measurements and assure the quality of their products. Similarly, the 9,000 calibrations performed by the NIST labs each year provide quality assurance through an even larger private-sector activity that disseminates transfer standards traceable to the national and international measurement systems. NIST's Standard Reference Databases provide evaluated, high-quality data that are used by many industries for improving the design of industrial processes, the quality of materials, and the performance of advanced information systems.
Because global market competition is becoming more technology intensive and trade is becoming a more significant determinant of the health of the U.S. economy, the nation's measurement infrastructure is growing in economic importance and strategic value. As we plan for the future, we ask ourselves three fundamental questions:
- What measurements and standards are needed by industry now, and what will industry need in the future?
- How good of a job are we doing in meeting these needs?
- How does our performance compare to the best in the world?
Our FY 1999 budget request directly answers these questions. New NIST initiatives included in the President's budget request address the multiplying needs for measurements and standards in semiconductor manufacturing, climate change and disaster mitigation technologies, and the international measurement system.
U.S. semiconductor sales were approximately $62 billion in 1996, with the industry growing at a healthy clip. Obviously, this is a critical sector of our economy. Right now, the U.S. semiconductor industry and its material and equipment suppliers are holding their own by a small margin as leaders in world market share, but they face major technological obstacles early in the next decade as the size of device features shrinks and chips become more complicated. At least 25 percent of the cost of semiconductor manufacturing is attributable to measurements, which is why the industry has made measurement challenges a key part of its technology roadmap--and why the roadmap's planners called upon NIST to play a major role in conquering those measurement difficulties.
We are living up to industry's expectations. Among our most recent accomplishments: a new measurement reference material that has the potential to leapfrog several generations of metrology needs forecast by U.S. semiconductor manufacturers. Developed by NIST and partners from SEMATECH and Sandia National Laboratories, this reference material is being proposed as a tool for calibrating equipment that measures integrated circuit features as tiny as 0.1 micrometer, or about a thousandth of the width of a human hair. Today's state-of-the-art commercial chips contain features approaching 0.25 micrometers. The new advance should enable the industry to continue its pursuit of ever faster and more powerful microchips.
The $2 million increase requested for NIST's semiconductor metrology work will permit us to continue to develop and deliver the most urgently needed new measurement tools and services in three areas: very thin insulating layers, lithographic metrology, and process control. This work will have a disproportionately large leverage because the nation's competitiveness in semiconductors is key to so many other sectors' success.
With nearly 23 percent of the world's carbon emissions, the United States currently leads every other nation, and this is almost entirely attributable to energy production and consumption,. In fact, the U.S. consumes almost twice as much energy per dollar of GDP as Japan and Germany. Implementation of advanced technologies can close that gap. Not only will this have environmental benefits, but implementation of these technologies through the year 2010 can potentially save U.S. industry between 70 and 90 billion dollars each year, making our economy much more competitive. Regardless of international agreements, reducing these emissions holds considerable promise for improving industrial efficiency.
NIST's unique role in this national and international effort focuses on assuring that the United States has the measurements and standards needed to assess any problems, take action on solutions, and determine the effects from new technologies. In FY 1999 NIST is requesting an increase of $7 million for measurement work and data efforts to help reduce industry's greenhouse emissions and to develop more efficient industrial processes. NIST plans to provide new measurement techniques and data to enhance biomass production and utility and carbon dioxide consumption, which could lead to removal of excess carbon dioxide from the atmosphere. We plan to develop databases, models and characterization techniques to help the chemical process industry convert from energy-consuming distillation-based separations to those using lower-energy membrane technology. We also plan to tackle the measurements and data needed by industry to develop energy-efficient and environmentally acceptable alternative working fluids for power generation, to develop replacements for refrigerants and insulating gases, to reduce perfluorocompound emissions during semiconductor production, and to enable industrial scale use of carbon dioxide to replace halocarbons used in chemical manufacturing and process industries. Our Manufacturing Extension Partnership centers around the country will help convey these new measurement tools, data, and models to the nearly 400,000 smaller manufacturers which want to improve their operations in environmentally sound -- and practical -- ways.
The natural disasters that are an increasingly regular fact of life cost the U.S. economy a billion dollars every week. That figure does not even include the indirect, but equally severe toll from the disruption of businesses and services that often follow such disasters. Heavy damage is inflicted on our nation's infrastructure systems, the basic installations and facilities which underpin the viability and growth of our communities. These include not only our buildings. They also affect the community's lifelines -- electric power, water, sewage, oil and gas delivery and storage, transportation, and industrial facilities. Moreover, they also include banking and financial services and fire safety and other emergency services.
These losses cannot all be avoided, but they can be reduced, and both public safety and our economic well being can be improved. In FY 1999, NIST is seeking an increase of $3 million to work on next-generation technologies and improved practices that will permit communities and technical experts to take cost-effective actions to reduce the effects of disasters and to improve response and recovery measures. NIST's unique role in this Department of Commerce initiative is in standards and measurements. Standards are a key means for getting the results of advances in science and technology into the hands of those who build buildings, install power, and communications and other critical parts of our infrastructure.
Our efforts will include developing and disseminating recommendations for better standards and codes for buildings and lifelines and communications systems. We also will be able to tackle new measurement methods to assess pre- and post-disaster conditions that will assist in strengthening critical structures, in making repairs, and in rapid recovery for businesses and communities. The initiative also will permit NIST to expand its work on detecting system vulnerabilities to natural disasters at an early stage, when there is still time to act rather than to simply react. When you consider the enormous costs of disasters, it is striking how little this nation is investing in technical approaches to reduce those costs. This initiative is a step forward for enhanced public safety.
The second major challenge that NIST faces now and into the next decade is to assure that measurement capabilities and standards are in place to support U.S. participation in global markets. The United States exports goods and services worth approximately $960 billion annually, making exports crucial to our nation's economy and providing jobs for Americans. We could be doing better. World trade has increased by 15 percent per year, compared to only 9 percent for U.S. exports. One reason our exports are not even higher is the technical barriers to trade that take the form of complex or inconsistent standards or specifications. Let me give you just one example. In 1996, the European Union introduced new requirements for electromagnetic compatibility. U.S. industry has found these requirements so difficult to comply with that several manufacturers have taken their products off the European market thus reducing U.S. exports. According to the Trade Promotion Coordinating Committee's 1996 report, in 1994 foreign requirements for testing, inspection and certification affected more than $150 billion of global U.S. exports.
NIST is seeking $4 million in FY 1999 to reduce barriers to exports by:
- developing measurement methods and the international infrastructure needed to ensure that measurements, particularly those needed to support new, high technology products are available worldwide,
- developing new international standards for emerging products that facilitate U.S. exports while seeking to harmonize standards, codes and regulations in collaboration with voluntary standards developers and U.S. regulatory agencies, and
- pursuing access to markets by training metrology experts from developing countries in U.S. standards, conformity assessments, and measurement practices.
For many physical and engineering measurements, we will work to further the establishment of a worldwide network of measurements traceable to NIST. This network is important for U.S. companies operating abroad and requiring NIST-traceable calibrations for regulatory purposes -- U.S. airlines and aircraft repair facilities, for example. The measurements needed are not merely the conventional measures of mass and volume, but must also include a broad array of technical measurements such as materials characterization techniques, electrical measurements with part- per-million accuracy, and information technology standards.
If we are to provide the United States with global leadership in measurements and standards, and if we are to open the international standards system, then NIST must have world-class facilities. In FY 1998, Congress provided $95 million for construction, renovation, and maintenance of NIST facilities. The release of $78 million was made contingent upon submission of a spending plan. A review of that plan should make it clear that U.S. industry and science urgently need improved facilities at NIST.
Our Advanced Chemical Sciences Laboratory is nearing completion, and should be ready for occupancy early next year, on time and on budget. The proposed Advanced Measurement Laboratory -- which will house NIST's most advanced metrology, physics, chemistry, electronics, engineering and materials research -- is ready to begin construction except for a relatively minor design update. In order to start construction in FY 1999, we need legislative language providing advanced appropriations. That will permit us to do "just-in-time construction," if you will, borrowing a proven private sector technique. We cannot wait to begin work on this critical facility. We are seeking a $40 million for the next increment in funding for the AML, with additional funds requested as advanced appropriations through FY 2002.
We have identified almost $400 million in facilities maintenance and capacity projects requiring major retrofitting. That is no surprise in light of our aging facilities infrastructure and previous studies. But it is urgent. In FY 1998, NIST plans to devote a total of $32 million toward these projects. The FY 1999 budget requests $16.7 million primarily for projects to ensure compliance with health, safety, environmental and access regulations. These include replacing major fire research emission control systems, asbestos abatement and fire alarm upgrades.
Our third challenge is to build greater consensus about the value of the Advanced Technology Program. Established to co-fund with industry the higher-risk, longer-range research and development that industry was unable or unwilling to support on its own, the ATP is intended to have broad economic impacts on our economy. Beginning eight years ago with just $10 million in awards, the ATP now has teamed up with industry in about 350 projects -- each with a long-term goal and considerable technical and commercial risk.
There are two reasons to be optimistic about the future of this program. First, we have seen a continuing flow of positive, early results -- and though all results of this program still must be considered preliminary and partial since the goal is to work on technologies that are five, seven, or even ten years from commercialization. We have always said that the value of the ATP should be judged by the ultimate success in creating wealth and jobs.
We have several years of interim results, and they are more promising with each new assessment. One economic study performed by an independent contractor projects a possible impact on the U.S. economy by the year 2000 of more than $3 billion due to quality improvements enabled by the ATP's 2-Millimeter Project, begun in 1992. This project involved a consortium of eight small and mid-sized companies, two universities, and Chrysler and General Motors, getting together to co-fund and enable development of technologies that allow U.S. auto manufacturers to routinely control the fit of auto body components to world-class levels. The results have been decreased production and maintenance costs, improved product quality, and reduced time to launch new models. Within the next four years, benefits are expected to exceed project costs many times over, and the benefits will flow broadly to the automotive sector -- one of our largest sources of jobs -- and to consumers. Plans are underway to further diffuse this technology to the manufacturers of furniture and appliances, where it also is expected to improve quality and productivity. These -- and other examples I cite later in my testimony -- are the kinds of broad-based benefits that ATP was designed to produce.
The second reason for growing acceptance of the ATP is the effort that we have made to strengthen the program. Commerce Secretary Daley listened to the concerns expressed by Members of Congress and made key revisions in the ATP. The revised procedures place increased emphasis on joint ventures and support for small and mid-sized firms. We also are more actively seeking to involve the states in the program and we are working to ensure that we are not competing with private capital. The House- passed authorization bill for NIST and the current Senate version both include the ATP.
We have begun nine new ATP competitions in FY 1998, with applications due over the next several months. Our current plans and funding would allow us to select approximately 80-85 industry-suggested projects this year. The proposed FY 1999 increase of $67.4 million above the current level is necessary to continue ongoing multi-year projects while permitting ATP to carry out a general competition and several additional focused program competitions. That would allow us to cofund about 90-95 new awards. We already have, and are continuing to receive, a number of good ideas from industry for new focused program competitions. The ATP is working, and this additional funding will permit us to continue this partnership with the private sector, investing in future technologies that could make a real difference in our economy.
I want to make one additional point about the ATP. We are experiencing very positive synergies with our laboratories' work. Having access to industry insights into cutting edge technologies is a real plus for our researchers; it provides them with critical advance information that allows NIST to prepare and refine our measurement priorities so that we can best meet industry's future needs. In a number of cases, our labs already have been able to provide measurement expertise to a variety of ATP participants. The NIST laboratories have benefitted in other ways from the ATP. A project by X-Ray Optical Systems is a perfect example. This small firm in Albany, N.Y., worked to develop powerful techniques to focus x-rays and neutrons into beams for use in research, medical imaging and treatment and manufacturing. One of the very first applications was in the NIST Center for Neutron Research, where it has dramatically enhanced our analytical abilities. The lenses developed by X-Ray Optical Systems have given a 100-fold increase in neutron intensity, and allowed us to perform analytical measurements on samples 100 times smaller than previously possible. This development might not have happened this decade if these special lenses had not been available so quickly. Without this collaboration, we would not yet have achieved the kind of results that our own researchers -- and dozens of guest researchers from companies, other government agencies and universities -- already have witnessed. Related work with X-Ray Optical also has given NIST researchers access to state-of-the-art x-ray lenses that put the capabilities of a massive synchrotron x-ray source in a lab-scale instrument. We've used this new capability to pioneer measurements of crystal structure and composition at scales and intensities that would previously have been accessible only with synchrotron radiation sources.
Our fourth challenge is to expand access to Manufacturing Extension Partnership services and reach more small and medium-sized companies, and to continue federal support for MEP centers after their sixth year of support. Since 1993 the MEP has grown from a promising, small pilot program operating in seven states to a national program in all 50 states accessible to the country's nearly 382,000 smaller manufacturers.
That's important, because smaller plants and factories account for almost two-thirds of all manufacturing jobs, employing about 12 million people at wages substantially higher, on average, than in the retail sector. These firms, typically employing fewer than 500 people, have generated about three-fourths of new manufacturing jobs created over the last two decades. They likely will become an even more crucial component of our economy as industry deals with the realities of a rapidly diversifying global marketplace and the need to involve smaller manufacturers up front in the manufacturing process.
The MEP's non-profit, locally managed centers are working hard, using lessons learned from the past, to attract and assist more manufacturers who want help in applying technical solutions to pressing problems. Since federal funding decreases after several years, the Centers have been especially mindful of the need to earn support from customer fees and other sources. Unfortunately, we know from experience that as the MEP centers seek to become more self- sustaining, they run the risk of underserving the smallest of manufacturers who typically need the centers' services the most. We are transmitting to Congress a new report that addresses this issue, which concludes that federal funding for these centers should continue at the one-third level after the first six years to ensure that these smaller companies can continue to receive the centers' services. We are hopeful that Congress this year will pass legislation that would allow us to continue funding centers after six years of operations if they pass a rigorous evaluation process. Up until now, we have continued funding those centers only due to actions by the appropriations committees -- which have made it clear that they want the House and Senate authorizing committees to deal with this issue this year. In both the House-passed authorization and the current Senate version, the needed changes are made. This is among the administration's highest priorities for NIST.
Our MEP budget request for FY 1999 decreases by $7.8 million, reflecting the lower federal share of the centers' operating costs since the federal share changes from one-half to one-third as the centers mature. The number of centers is not expected to change, and we will be concentrating on integrating the centers and taking advantage of best practices to extend our reach to more manufacturers and to do so in the most cost-effective manner.
There is perhaps no other effort in the entire federal government with greater leverage than the Baldrige National Quality Program. It has inspired thousands of U.S. companies to improve their products and services and to expand their markets through performance excellence. Earlier this month, more than 1600 representatives from around the country-- and several dozen overseas visitors -- gathered in Washington just to hear more detail about the performance excellence strategies of the Baldrige Award 1997 winners. That is the largest turnout ever at this annual information-sharing event.
Our program has been strengthened by the establishment of state and local quality programs that mirror the Baldrige quality management guidelines. This grassroots and regional growth has provided NIST with another nationwide network through which we can share information about improving company performance and competitiveness.
The fifth challenge for NIST in the years ahead is promoting performance excellence in healthcare and education, using the Baldrige Award approach. Both the education and healthcare sectors have expressed very strong interest in having NIST establish award categories. Today, only for-profit organizations can apply for a Baldrige Award and benefit from the formal process of expert reviews. We ran highly successful pilot programs in both the healthcare and education areas in 1995, and they were extraordinarily popular. We were not successful in persuading Congress to fund these awards in FY 1998. I think this is a significant missed opportunity for our economy and our society. The Foundation for the Malcolm Baldrige National Quality Award -- which endowed the business award with more than $10 million when it began in 1987 -- has committed to raise $15 million for healthcare and education awards if the federal government does its share, which would be just $2.3 million under the FY 1999 request. The Foundation has raised nearly $2 million in pledges already. Every industry speaker at our recent 10th anniversary Baldrige ceremony, and most of the CEOs at our recent Quest for Excellence conference, made a special point about the need to move ahead now with these awards, and we are hopeful that we can get authorization bills in the House and Senate that meet that need. The current Senate authorization bill adds both new award categories; the House-passed bill would permit us to add one new program each year for the next two years. There is enormous potential for impacts on our economy and our health and education.
I said earlier in my testimony that NIST is results-driven, and that we put considerable energies into evaluating and refining our programs. In fact, I feel comfortable in asserting that we are clearly among the leaders in the federal science and technology establishment in terms of evaluation and review of results. As I told my agency's staff recently, we are the institution known for measurements, and we should be able to lead the way when it comes to performance metrics. Our budget submission to this subcommittee provides details, but I would like to briefly summarize our approach and some results to date.
We make extensive use of peer reviews. Each of our laboratory programs is reviewed by panels convened by the National Research Council which look at the labs' plans and projects. We recently established a national advisory board to review the Manufacturing Extension Partnership program, with experts from industry, state and local governments, and universities. Our Baldrige Program is scrutinized in detail by a Board of Overseers, and NIST as a whole has the advantage of oversight and guidance from a long-standing Visiting Committee composed of private sector experts from a variety of organizations. Each of these organizations has provided insights that have fine-tuned our planning and affected our decisions on program management. These committees are not shy in identifying areas in which they believe NIST can have a greater impact. We hope to soon have a similar review structure for the Advanced Technology Program, which will supplement the current external reviews of proposed projects and focused programs and the Visiting Committee's monitoring of the ATP.
NIST has long been a leader in the use of economic impact studies of R&D programs carried out mostly through commissioned contractors and consultants. These studies provide both qualitative assessments and quantitative estimates either as benefit-cost ratios or as social rates of return. The results consistently show that rates of return from NIST laboratory research are high compared with private investments in technology and other public technology investments. According to several of these studies, the benefit-cost ratio for our measurement and standards research programs has ranged from 3 to 1 to more than 100 to 1. In other words, for every dollar spent on NIST research in the studied areas, the economy realizes benefits of $3 to more than $100. A good example is a recent analysis of our work to develop 28 reference materials needed to improve the reliability of radiopharmaceutical dosages for medical diagnostics and treatments. In this case, our benefit-to-cost ratio was 98 to 1 with a social rate of return of 138 percent.
These results are so large because NIST targets its research at specific infrastructure problems faced by a large number of firms or industries which have been identified and prioritized through cooperative strategic planning with industry. Broad industry participation promotes rapid acceptance and wide diffusion of the research results, thereby leading to high rates of return. These assessments not only measure and analyze current and past performance; they are important guideposts for future planning.
Consulting with leading economists, our Advanced Technology Program has developed a strong evaluation strategy, one which is unique among federal R&D efforts to our knowledge. Highly qualified academic and consulting economists and other evaluation experts assist in- house ATP staff in planning, modeling and developing databases, and in conducting surveys, case studies, statistical and econometric analyses and other studies to enhance program performance and the quality and reliability of performance measures.
The ATP's novel Business Reporting System encompasses all projects and participants funded by ATP since 1993. Third-party studies are conducted periodically. Among our latest analyses is a comprehensive study of over 200 projects funded from 1993-95 which shows that the ATP is having a significant impact in accelerating the R&D process, stimulating high-risk and enabling technology development, and reducing the time-to-market in highly competitive markets. For example, 86 percent of ATP-funded organizations indicate that they have accelerated their R&D cycle as a result of ATP funding. Companies surveyed already have identified more than 1,000 potential applications for the ATP-supported technologies and have developed commercialization plans for nearly 800 of those technologies.
In addition to its new advisory board, the Manufacturing Extension Partnership measures performance and conducts evaluations in a variety of ways, including an ongoing survey of client firms by the Bureau of the Census. For example, based on a recent sample of 2,350 client companies around the country, MEP estimates that in FY 1997 alone, MEP services generated approximately $214 million in new sales, $31 million in inventory savings, and $27 million in labor and material savings -- and leveraged $156 million in additional capital investment by client firms. These are very conservative assessments, because they do not take into account the multiple year benefits from the companies' improvements. MEP management scrutiny of each center includes monthly progress reports of activity, outputs, and outcomes, an annual review of each center's operations, and legislatively mandated third-year reviews of each center by an external review panel. When all is said and done, it is the results of individual technical assistance projects that best quantify the impact of MEP, and every center is racking up success stories as they serve more manufacturers. One of my favorites is Baron Industries of Dalton, Georgia. This manufacturer of glass range tops received technical and business advice provided by the Georgia Manufacturing Extension Alliance. By following the MEP affiliate's recommendations in revising plant layout and work processes, Baron saved more than $800,000 in inventory, overtime pay, equipment, and materials costs. At the same time, they boosted labor productivity by 40 percent, reduced the internal scrap rate from 80 percent to 10 percent -- and added 16 new jobs.
In addition to the Baldrige National Quality Award's prestigious Board of Overseers, the Baldrige program relies on several other evaluation mechanisms to assure responsiveness to the needs of industry and overall effectiveness. These include internal metrics covering the use of the criteria documents and an annual improvement day where experts offer reviews about the effectiveness of the program and the specific award criteria. Ultimately, the program's effectiveness is evaluated by its impact on the performance of award participants. Recently, and for the fourth straight year, a NIST analysis concluded that an index of Baldrige Award winners outperformed the S&P 500 by about 3 to 1, a figure that is especially impressive during a period of strong stock performance. We have a growing body of evidence that Baldrige Award recipients demonstrate major gains in productivity, cost reduction, market performance, and competitiveness. For example, improvements of 20 to 70 percent in productivity over five years are common. The numbers show that following the Baldrige approach pays off.
I want to offer a final word about performance evaluations and metrics. We believe that undertaking assessments and doing careful measurements of the effectiveness of NIST programs are both necessary and helpful in better understanding our work, in guiding us to deliver the greatest value for industry and the taxpayers, and in making decisions about future directions. The Government Performance and Results Act of 1993 requires agencies to be more thoughtful and objective about the effectiveness and impacts of taxpayer-funded programs. Since we have considerable experience in evaluating our work, we also know that there are many pitfalls in trying to evaluate R&D type activities, especially those in which the results are dispersed throughout the economy and in which benefits may not appear for years after the initial research was conducted. Our simple advice, perhaps, is that "policy makers beware" when reviewing all evaluations and metrics, and understand the limitations of what is still more of an art than a science. As in many areas of life, we all need to look beyond the numbers and to ask the tough questions.
Having said that, I thank you, Madam Chairwoman, and I would be happy to answer any questions the Subcommittee might have.