[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[comments-gtlds] InterNIC Midterm Evaluation and Recommendations




         InterNIC Midterm Evaluation and Recommendations


         A Panel Report to the National Science Foundation


                         December 1994



                            Prue Adler
                      Assistant Executive Director
                 Federal Relations and Information Policy
                    Association of Research Libraries


                           Kilnam Chon
                             Professor
                      Korea Advanced Institute of
                        Science and Technology


                           Daniel P. Dern
                          Internet Analyst


                           Alan Emtage
                   V.P. for Research and Development
                    Bunyip Information Systems, Inc.


                           Peter S. Ford
                     Member of the Technical Staff
                    Los Alamos National Laboratory


                          Joan C. Gargano
            Director, Distributed Computing Analysis and Support
                        Information Technology
                     University of California, Davis


                        Martyne M. Hallgren
                          Executive Director
              GateDaemon Consortium, CU-SeeMe Consortium
                          Cornell University


                         Anthony C. Hearn
                 Resident Scholar for Information Sciences
                        The RAND Corporation




                        Ellen S. Hoffman
                      Development Director
              Merit Network, Inc./University of Michigan


                          Karl Jacob
                      Chief Technical Officer
                         On Ramp, Inc.


                        Michael Mealling
                  Network Information Coordinator
                  Office of Information Technology
                   Georgia Institute of Technology


                        Barbara Morgan
                Director, Strategic Technology Planning
                 University of California at Berkeley


                         Mark Resmer
                     Associate Vice President
                      Information Technology
                     Sonoma State University


                     Anthony M. Rutkowski
                        Executive Director
                         Internet Society


                       Gleason Sackman
                      Coordinator, SENDIT
                North Dakota's K-12 Telecom Network


                      Michael F. Schwartz
                 (Panel Chair and Report Editor)
                       Associate Professor
                   Computer Science Department
                  University of Colorado at Boulder




Contents



   1   Executive Summary  : : : : : : : : : : : : : : : : : : : : : : : : :
2
   2   Introduction  : : : : : : : : : : : : : : : : : : : : : : : : : : : :
5
       2.1    Background  : : : : : : : : : : : : : : : : : : : : : : : : :
5
       2.2    Mechanics of the Review Process  : : : : : : : : : : : : : : :
5
       2.3    Capsule Summary of Awardees' Agreed-Upon Services : : : : : :
6
       2.4    Official Status of this Report   : : : : : : : : : : : : : : :
8
   3   Changes in the Internet Since the Solicitation   : : : : : : : : : :
9
   4   InterNIC Directory and Database Services - AT&T : : : : : : : : : : :
12
       4.1    Directory of Directories (Yellow Pages) : : : : : : : : : : :
12
       4.2    White Pages  : : : : : : : : : : : : : : : : : : : : : : : : :
13
       4.3    Database Services : : : : : : : : : : : : : : : : : : : : : :
14
       4.4    Administration of Services   : : : : : : : : : : : : : : : : :
14
   5   InterNIC Information Services - General Atomics  : : : : : : : : : :
15
       5.1    Overview : : : : : : : : : : : : : : : : : : : : : : : : : : :
15
       5.2    Overall Management  : : : : : : : : : : : : : : : : : : : : :
15
       5.3    Service Evaluation : : : : : : : : : : : : : : : : : : : : : :
17
       5.4    IS Self Evaluation  : : : : : : : : : : : : : : : : : : : : :
22
       5.5    Evaluation Summary and Recommendations For Actions
              to the NSF   : : : : : : : : : : : : : : : : : : : : : : : : :
22
   6   InterNIC Registration Services - Network Solutions, Inc. : : : : : :
28
       6.1    Service Overview   : : : : : : : : : : : : : : : : : : : : : :
28
       6.2    Evaluation and Recommendations  : : : : : : : : : : : : : : :
28
   7   NSF Meta-Questions  : : : : : : : : : : : : : : : : : : : : : : : : :
31
       7.1    Overview  : : : : : : : : : : : : : : : : : : : : : : : : : :
31
       7.2    NSF Management of the Cooperative Agreements   : : : : : : : :
32
       7.3    Requirements for Ongoing ``Customer'' Feedback  : : : : : : :
33
       7.4    Appropriateness of Informal Relationship between
              the InterNIC and CNIDR   : : : : : : : : : : : : : : : : :
34
       7.5    The Viability of the NIS Manager Concept  : : : : : : : : : :
34
   A   InterNIC DS's Responses to Panel Questions  : : : : : : : : : : : :
36
   B   InterNIC IS's Responses to Panel Questions : : : : : : : : : : : : :
41
   C   InterNIC RS's Responses to Panel Questions  : : : : : : : : : : : :
47
   D   List of Acronyms : : : : : : : : : : : : : : : : : : : : : : : : : :
50



                                 1




1   Executive Summary


In 1992, the National Science Foundation (NSF) awarded three five
year cooperative agreements, to American Telephone and Telegraph
(AT&T) Company, General Atomics (GA), and Network Solutions, Inc.
(NSI), to provide a Network Information Services Manager
supporting Directory and Database Services, Information Services,
and non-military Registration Services (respectively) for the
NSFNET. The awardees adopted the name InterNIC for their joint
role.  The solicitation included provisions for an external
review after 24 months of operation, which resulted in the
current report.

This report provides an evaluation to the NSF about the
performance of the awardees, and a set of recommendations to the
awardees about how they might better meet the needs of the U.S.
Research and Education community during the remainder of the
award period.  The review was based on how well the awardees have
provided their agreed-upon services, rather than on some other
(possibly updated) characterization of the appropriate role of a
Network Information Services Manager.

The report presents recommendations from a review panel convened
by the NSF to represent a broad range of expertise and interests.

In the evaluation of the Database and Directory Services InterNIC
component, the panel found that AT&T has provided all of the
services specified in their cooperative agreement, and has
performed admirably given the rapidly changing nature of
information tools that have been introduced since the start of
the InterNIC. The panel's critiques of AT&T's InterNIC component
concerned insufficient community outreach to populate their
directory-of-directories, and the use of software that is
difficult to use during searches and registrations of both white
and yellow pages services.  The panel's recommendations for AT&T
concern the longer-term appropriateness of a centralized
directory-of-directories model, and the need to focus their white
pages service efforts on fostering widespread adoption of
distributed user white pages technology.

In its evaluation of the Information Services InterNIC component,
the panel found that GA has lacked leadership and direction, and
has not satisfactorily provided most of the services specified in
their cooperative agreement.  The panel notes that GA has
provided a small number of very good services, primarily as the
result of two specific individuals (Susan Calcari and Kent
England), who have accomplished their work apparently despite the
management of the project rather than because of it.  The



                                 2




project's alarming rate of turnover, recent reorganization
(placing into power what the panel perceived as a manager with
inappropriate vision and insufficient community awareness), and
inappropriate prioritization of its various efforts left the
panel feeling that the current arrangement should not be
continued.  The panel had a split of opinions on what to
recommend to the NSF. The majority (13 of 16 panelists) felt that
the cooperative agreement with GA for the provision of IS should
be terminated as soon as possible, and that the NSF should find a
way to continue to support specific projects that the panel
deemed as important and viable.  Three minority opinions arose,
two of which suggested continuing funding GA for portions of the
project (with disagreement about which portions to continue), and
one suggesting that the NSF continue funding a somewhat reduced
set of functions over a probationary period, after which time
another review be held to assess the situation.  While the panel
disagreed on these recommendations, there was general agreement
that the NSF should continue to support the InfoScout, in the
person of Susan Calcari.

In its evaluation of the Registration Services InterNIC
component, the panel found that NSI has provided an excellent
service in the face of exponential growth in demand, and that
they have adapted to this demand appropriately by delegating and
distributing the registration activities in most cases.  The
panel also recognized NSI's success in fostering
internationalization of RS functions, as well as the Information
Sciences Institute's success in administering .US domain
registrations as a subaward under NSI. The panel's
recommendations included continuing to move the WHOIS legacy
database function to more appropriate distributed technology, and
moving to automate registration processing as much as possible.
The panel also recommended that NSI begin charging for .COM
domain name registrations (which pose the most significant growth
problems) as soon as possible, and eventually charge for all IP
registration services.  The panel further recommended that the
NSF be prepared to allocate more resources to NSI to help them
meet their increasing work load.  Finally, the panel encouraged
the NSI and the NSF to develop suitable legal protection of the
registration function, in response to NSI's experiences with
companies' asserting trade and service mark claims in .COM domain
name registrations.

As part of the review process, the NSF asked the panel to answer
a set of ``meta-questions'' about the viability of the Network
Information Services Manager concept.  The panel felt that



                                 3




successful InterNIC services are important and should be
continued, but that the current efforts are insufficient and in
fact that no single program could meet the full spectrum of
needs.  The panel suggested in particular that the NSF consider
ways of building Internet information services from both the top
down and the bottom up, similar to how it so successfully helped
to develop the physical network infrastructure.  The panel also
suggested that the NSF consider more focused awards of a shorter
duration, and that NSF require specific methods for user feedback
rather that the unsolicited individual user electronic mail
offered by the InterNIC.

The panel was also asked to consider the appropriateness of the
informal relationship between the InterNIC and the Clearinghouse
for Networked Information Discovery and Retrieval (CNIDR). The
panel felt that the liaison has been beneficial (particularly for
InterNIC Directory and Database Services) and should be
continued, but that it is not advisable to complicate
organizational relationships with formal inclusion of CNIDR in
the InterNIC.

Given GA's problems, the panel felt it important to consider the
NSF's oversight role in the project. 1  The panel felt that the
NSF has been more successful in its work to develop the physical
network infrastructure than it has in developing Network
Information Services.  The InterNIC awards set the precedent of
requiring significant self-coordination among a team of awardees,
and requiring outreach to other Network Information Centers.  The
panel suggests that the NSF critically consider whether it is
viable to expect significant self-coordination among a team of
awardees in future awards.  The panel also notes that the NSF's
program management was not able to correct GA's problems early on
despite excellent efforts by the NSF staff, primarily because the
NSF staff were overextended by monitoring at least two major
projects (the InterNIC and the NSFNET backbone) at once.  The
panel recommends that for future large scale efforts in the
rapidly changing Internet environment, the NSF should form an
ongoing advisory panel of outside experts or employ some external
consultants to help manage such cooperative agreements, rather
than waiting two years to call for a review.



____________________________
   1 Note: Peter Ford was asked to leave the room during this part of the
panel
   discussions, because of his part-time consulting relationship with the
NSF.



                                 4




2   Introduction


2.1   Background

In response to the rapid growth of the NSFNET and the expiration
of the funding arrangement for the NSFNET National Service Center
(NNSC), in the Spring of 1992 the U.S. National Science
Foundation (NSF) issued Project Solicitation NSF92-24, calling
for one or more Network Information Services Managers (NIS
Manager(s)) to provide and/or coordinate (i) non-military
Registration Services (RS), (ii) Directory and Database Services
(DS), and (iii) Information Services (IS) for the NSFNET. The
proposals were reviewed by an external panel in June 1992, after
which time the NSF selected a set of awardees and negotiated five
year cooperative agreements totaling over $12 million.  The
awards began January 1, 1993, and were required to reach full
operation by April 1, 1993.  The solicitation included provisions
for an external review after 24 months of operation.  The current
report represents the evaluation and recommendations produced by
that review.

The purpose of the current review is twofold.  First, the review
should provide an evaluation to the NSF about how well the
awardees have met their goals, and a set of recommendations about
what should be done for the remaining 3 years of their
cooperative agreements.  Second, the review should provide
recommendations to the awardees about changes that would better
allow them to serve the U.S. Research and Education (R&E)
community.



2.2   Mechanics of the Review Process

The current review panel was selected by the NSF to represent a
broad range of expertise and interests of the R&E community, but
also included representation from the international community and
the commercial sector.  The membership of the current panel
(hereafter referred to simply as ``the panel'') is mostly
independent of that of the original proposal review panel,
although for the sake of programmatic continuity there was some
overlap between the panels.

The panel met November 16-17, 1994, after having received and
reviewed a set of materials from each of the awardees.  During
these two days the panel heard presentations from each of the
awardees.  After each presentation, the panel held a closed
session, discussed the presentations, and constructed a set of



                                 5




questions for the awardees to answer.  These questions and
answers are contained in appendices of this report.  The current
report resulted from detailed discussions among the panelists
both during and after the meeting.

Because the awardees were selected by a prior review process that
considered how well the proposal fit the solicited functions, the
panel decided that the evaluation should consider only how well
the awardees provided the services specified in their cooperative
agreements (outlined in Section 2.3 below), without further
analyzing how well it met the goals of the original solicitation.

A complicating factor in the review process is the fact that the
Internet community, hardware, and software technology have
evolved considerably since the time of the initial award.  The
panel decided that the evaluation should only consider these
changes when the awardees needed to adapt in order to provide
their agreed-upon services.  In addition, the panel considered
Internet changes in its recommendations to the awardees about
changes that would better allow them to serve the R&E community.



2.3   Capsule Summary of Awardees' Agreed-Upon Services

In brief, the awardees under review agreed to provide the
following services in their negotiated cooperative agreements:


   o American Telephone and Telegraph (AT&T) Company provides the
     DS portion of the award.

   o General Atomics (GA) provides the IS portion of the award.

   o Network Solutions, Inc.  (NSI) provides the RS portion of
     the award.


Together, these three awardees adopted the name ``InterNIC''.
Below we provide excerpts from a press release issued jointly by
the InterNIC awardees in January 1993, primarily to provide the
reader a brief sense of the InterNIC's goals.  A complete list of
goals is listed in the original proposal, available from the NSF.
In the remainder of this report we assume the reader has read the
proposal and hence is familiar with the services the InterNIC was
responsible for offering, and the motivations for these services.


AT&T agreed to:



                                 6




   o ``...  develop and maintain a Directory of Directories,
     including lists of FTP (File Transfer Protocol) sites, lists
     of various types of servers available on the Internet, lists
     of white and yellow page directories, library catalogs and
     data archives.''

   o ``...  provide white and yellow pages type Directory
     Services.  Access to these services will initially be
     provided through several currently popular in-use interface
     methods while migrating to the use of X.500 technology...''

   o ``...  [provide] database services, [including] the
     establishment of Database Services to extend and supplement
     the resources of the NSFNET, such as databases of
     contributed materials of common interest to the user
     community.''

   o ``...  offer database design, management, and maintenance to
     institutions and groups for inclusion in the Internet.''

GA agreed to:

   o ``...provide Information Services acting as the Network
     Information Center (NIC) of first and last resort and the
     NIC of NICs.''

   o ``...  include a full-service Reference Desk, a database of
     comprehensive networking materials called the Info Source,
     training classes and documentation, and coordination
     services among all appropriate groups in the community.''

   o ``...  [implement] several new approaches to distributing
     services [including] NICLink, a user-friendly hypermedia
     interface offering access to the Info Source and all the
     information it contains...  and the Info Scout, an
     individual who will scout out new resources and innovative
     uses of the network for inclusion in the Info Source.''

In addition, although it was not included in the press release,
GA had agreed to coordinate activities among the InterNIC
partners.


NSI agreed to:

   o ``...  provide registration services as the IP registrar.''



                                 7




   o ``issue IP numbers worldwide using delegated registries
     under the guidance of the Internet Assigned Numbers
     Authority.''

   o ``register domain names, and track points of contact.''

   o ``...  [provide the information from these assignments] to
     the directory and database services provider to be made
     available to the entire Internet community.''

   o ``periodically release the top level zone files to be used
     by all root Domain Name servers.''



2.4   Official Status of this Report

This report presents a set of panel recommendations.  The
National Science Foundation may choose to implement the
recommendations operationally as it deems appropriate, and in the
best interests of the program.



                                 8




3   Changes in the Internet Since the Solicitation


The Internet community, hardware, and software technology have
evolved considerably since the time of the initial NIS Manager
award.  In this section we summarize some of the more salient
changes, as background for recommendations to the awardees about
changes that might better allow them to serve the R&E community
in the future.

The most significant change in the physical architecture of the
Internet is the move from an NSF-funded backbone network
providing connectivity between regional and national Internet
service providers, to NSF-funded interconnection points for these
same providers.  The current NSF strategy is driven by the rapid
growth in numbers of commercial Internet service providers in
response to the rapid growth of the Internet market.  The good
news for the R&E portion of the Internet is that this growth has
resulted in growth in the quantity and quality of available
NIS's.  There are for-profit and non-profit organizations
offering IP connectivity services, and most have NIC or marketing
staffs.  There are more specialized NICs than ever before.  Both
the U.S. National Aeronautics and Space Administration (NASA) and
the U.S. Department of Energy (DOE) continue to expand their own
NICs.  Others are just starting up.

The number of commercial organizations providing training,
documentation, consulting, or delivering some product via network
information tools is rapidly increasing, fueling a highly
competitive market place.  Universities are examining new support
models for help desks and consulting groups as applications focus
on distributed computing and departments hire internal support
people.  Funding agencies such as the NSF and the Department of
Education require ``support'' as part of approved proposals.

Groups who do not offer connectivity services have also entered
the NIS world.  These groups, to name just a few, include public
libraries and community groups such as SIG-WEB. SIG-WEB is an
independent special interest group based in the San Francisco
area.  It draws representatives from a diverse group of
organizations including the telecommunications industry,
education, government, universities, and public institutions.

New tools such as Gopher, the World Wide Web (WWW), and Mosaic,
give users control over access to their own information as well
as to other resources on the network.  These tools are evolving
rapidly.  At the time the InterNIC proposal was written, the WWW



                                 9




was just becoming known in the technical community involved with
developing Networked Information Discovery and Retrieval tools.
Today the WWW is one of the most visible aspects of the Internet,
and it would not make sense to run a NIC without offering a WWW
interface.

The number of forums and meetings focusing on NIS's have
increased.  The Internet Engineering Task Force (IETF) has an
energetic User Services Area.  FARNET has an annual meeting
focused on NIS topics.  SIGUCCS has a Networking Task Force, in
addition to two annual services meetings.  Regional networks have
their own user conferences (for example, NYSERNet, NorthwestNet,
and MORENet).  The InterNIC offers training as well as a NICFest.
There is the annual European Network Services Conference plus
many other conferences in the U.S.

Finally, over the past few years a large number of books,
magazines, commercial courses, CD-ROMs, online materials, videos,
and consultants focusing on the Internet have become available.
For example, there are currently at least 150 Internet-related
books available from commercial publishers, with anywhere from
another 50 to 100 known to be upcoming.

These changes are significant to the goals of the InterNIC in a
variety of ways.  The growing number of NICs and commercial
training/consulting organizations implies an increasing need for
InterNIC outreach activities.  Rapid change in networked
information tools requires the InterNIC to upgrade their tool
base regularly, both to provide more capable service and so that
the InterNIC will not be viewed as ``outdated''.  The increasing
number of NIS-related forums implies that the InterNIC needs to
target their outreach efforts carefully to reach those audiences
that are most appropriate and receptive.  Finally, the increasing
number of Internet-oriented books and related materials implies
that the InterNIC should not be in the business of creating such
content.

As will be discussed in this report, the InterNIC has adapted to
these changes to varying degrees of satisfaction.

More generally, the continuing commercialization of the Internet
has major repercussions for NIC services.  On the one hand, there
is more information available in many forums -- online, in print,
and from individuals (both professional and volunteer).  This
information is not of uniformly high quality, and it is
increasingly difficult for many users to assess the quality of
the information they find.  At the same time, the overall effect
of the changing Internet is a greater reluctance to share
information because of its perceived commercial value.  This also



                                10




means that there is a greater need for the InterNIC to help the
flow of accurate information, and in particular to assist the R&E
community, which cannot always rely on commercial interests to
meet their specific requirements or to be objective in their
assessments.  This is a marked change from the period in which
the solicitation was issued, when a smaller Internet community
had a greater degree of shared value for such cooperative efforts
and a greater focus on R&E requirements.  The rapid growth of the
Internet beyond the R&E community means that the InterNIC needs
to be constantly reassessing its primary mission and the best
ways to meet the challenges raised by the rapidly changing
environment.



                                11




4   InterNIC Directory and Database Services - AT&T


The panel congratulates AT&T on a job well done.  AT&T has
provided each of the services defined in their cooperative
agreement, contributed some innovative technical solutions in the
areas of yellow pages and white pages, and made good use of
Internet information tools at a time of rapid change in these
technologies.  We feel the Directory and Database Services
portion of the InterNIC provides a useful service to the R&E
community.

We provide critiques and suggestions about each of the DS
components below.



4.1   Directory of Directories (Yellow Pages)

The panel feels that AT&T has built the current incarnation of
this service using appropriate technology (WWW and WAIS). We
offer critiques in two areas:  the shorter-term problem of the
size and usefulness of the database, and a longer-term issue of
the appropriateness of the current model.

At present, the service receives 4,000 queries per day.  While
significant, this query rate is an order of magnitude lower than
the typical query rates of other well-known, widely popular
networked information services.  We believe there are two main
causes.  First, although the underlying technology is reasonably
functional and stable, the interface is difficult to use.  The
user is given limited instructions on how to build a search, of
the form ``You may use compound searches, such as ...''  The
panel recommends enumerating the possible types of compound
statements, and providing some hints on how to improve searches
when a search fails.  It would also be useful to provide limits
on the number of hits returned, and to allow users to narrow
searches interactively.

Second, because the service contains only 700 entries, it does
not offer enough useful data to attract widespread use.  We
believe this paucity of entries can be traced to the difficulty
of entering a directory into the service.  At present users must
read a lengthy explanation and then send an electronic mail
message to enter data.  The panel recommends that AT&T reduce the
difficulty of entering information into the database, by
providing a WWW form as an option for entering the data, and by
reducing the amount of information that users must enter.



                                12




The longer-term issue we wish to raise is the appropriateness of
a centralized directory-of-directories model.  While a
centralized database provides a convenient locus for
registrations and queries, as the R&E community deploys
increasing numbers of databases, this approach will have scaling
problems.  Moreover, even with the official status afforded by
being the InterNIC DS, there is no reason why a particular
directory-of-directories should be expected to become widely
accepted as the authoritative source for such information.
Indeed, there are many other projects providing such
functionality, including a variety of distributed indexing
services and many WWW pages that provide comprehensive lists of
pointers to databases around the Internet.  The indications are
that such directories will become increasingly numerous over
time, particularly given the increasingly commercial nature of
the Internet, and the value to be provided by offering such a
service.  The panel recommends that AT&T again consider a more
distributed approach to providing the directory of directories
function, by continuing to work with R&D efforts under way in the
IETF and elsewhere.



4.2   White Pages

To date AT&T has provided all of the white pages services
specified in its cooperative agreement on machines located within
company premises.  While this approach has worked satisfactorily
to date, over time scaling problems will demand a much more
distributed solution.  We recommend that during the remainder of
the cooperative agreement AT&T focus its white pages service
efforts primarily on the increasingly critical task of building a
distributed white pages directory of users, with the goal of
making every U.S. R&E user accessible by name, organization and
discipline.  Such a service should deal gracefully with the
situation where particular queries may result in thousands of
hits, allowing users to narrow the search scope iteratively.
Along with this technology, AT&T should establish a policy
regarding organizations that do not want to provide their
information or that only wish to provide partial information.  We
recommend that AT&T work with other groups in establishing this
policy, including the North American Directory Forum (NADF) and
the IETF.

In order to help foster the widespread adoption of appropriate
distributed white pages technology, we recommend that AT&T



                                13




identify sites that have successfully implemented campus-wide
white pages services (such as the University of Michigan's X.500
LDAP deployment and the University of Illinois' CCSO system) and
then work with the R&D community (and possibly with commercial
service providers) to develop software distributions that will
permit sites to install user white pages services with modest
effort, and that allow interoperation among several popular white
pages systems.  The panel feels that AT&T should exploit its
position as InterNIC DS provider to foster widespread adoption of
such important technologies in the U.S. R&E community, and that
if implemented well these services could spread to commercial
services, such as AOL and CompuServe, and gain momentum similar
to the way Gopher and WWW have.

Part of the effort needed to make this happen is to develop tools
that allow sites to populate a site's white pages database from
information available in other common database formats.  Doing
this will require some background research to determine, for
example, what database systems are in common use for university
student registration systems, and then developing and packaging
the needed conversion software.  The other critical component is
to focus on outreach, rather than on providing white pages
services on centralized machines located within AT&T premises.



4.3   Database Services

AT&T has provided the database services specified in their
cooperative agreement, with inputs from the NSF program
management.  At the review meeting, AT&T agreed that this element
of the offered services was of a lower priority than the
Directory Services element.  The panel sees a continued need for
this service in order to support databases important to parts of
the R&E community that lack resources to host such databases.
The panel wishes to reinforce the intention that these databases
be given permanent homes elsewhere as soon as it becomes
feasible.



4.4   Administration of Services

The panel commends AT&T for the excellent work in the
administration of their services.  The availability of each of
the servers is exceptional in the light of dealing with
technologies that are not completely stable.  The rapidly
changing landscape of Internet information tools makes AT&T's
achievements even more noteworthy.



                                14




5   InterNIC Information Services - General Atomics


5.1   Overview

     ``The successful provision of information services
     during the next five years of explosive growth will
     depend on several important factors.  These include the
     ability to stay in touch with the community and
     recognize important service needs and future trends; the
     ability to remain flexible and adapt quickly to changes
     in the network offerings and in the user base; the
     successful implementation of distributed services to the
     midlevel and campus NICs and to end users; and the
     establishment of a strong leadership role in the
     community.''

     Network Information Services Manager for NSFNET and NREN
     Information Services, California Education and Research
                        Federation Network
                             May 1992

While a small number of the services developed by GA for the
InterNIC have contributed to the usability of the Internet, they
have not adequately provided the majority of services specified
in their cooperative agreement.  It appears that many of the
failures can be traced to problems associated with the management
of this part of the InterNIC by GA. Conversely, the successes
appear to be the result of specific individuals on the staff,
notably Susan Calcari and Kent England, who have accomplished
their work apparently despite the management of the project
rather than because of it.



5.2   Overall Management

The failure of the Network Information Services group to meet the
requirements of the cooperative agreement can be directly related
to the project management provided by GA. The loss of one of the
original Principal Investigators created a major disruption in
the initial progress of meeting the cooperative agreement
requirements.  It created an ongoing lack of continuity in
direction that was not resolved by the remaining co-principal
investigator.  It took eight months to install a new PI (Kent
England) to provide the needed leadership.  He has made good



                                15




progress in identifying problem areas and refocusing the group on
activities in which they have a competence and strength.  Even
so, England's time and attention were only partially focused on
the InterNIC project, as he was also responsible for the
management of CERFNet.

Unfortunately, several of the areas are still not being addressed
sufficiently, and GA has decided to reorganize the IS group and
again install new management.  In attempting to assess the
prospects for ongoing success under the new management of Karsten
Blue, the blandness and uninformed quality of the strategic
mission for the project at the senior management level is quite
discouraging.  To provide information about the Internet to the
American public as the entire strategic mission statement for the
project is hardly sufficient, and speaks to the limited vision of
the new management.  This comes at a time when the group has
barely begun to reestablish itself in areas of core competencies,
and has not even had the chance to make progress in new areas.

The new management has no experience in providing network
information services, and through the panel review process, the
new management did not demonstrate a comprehension of the current
Internet environment or a vision as to how the IS should evolve
and be provided during this evolution.  Considering the rate of
change in Internet growth and service requirements, this lack of
experience or leadership capabilities in this area severely
compromise the ability of GA to perform at an acceptable level,
let alone improve in areas that need attention.

The continuing, alarmingly high rate of personnel turnover
throughout the project team suggests that whatever factors are
behind these problems remain unresolved.  It is not clear how GA
intends to address the causes of staff attrition, and thus
whether the company is in a position to provide a stable
environment within which the project could flourish.  Associated
with this issue is a concern that Kent England's departure leaves
the quality of technical and strategic leadership of the project
unknown.

GA has positioned the IS team in the role of direct provider of
all services, rather than a promoter and facilitator of
designated services by others.  Not only has this caused the
InterNIC to compete with other service providers in such areas as
the delivery of end-user workshops, but it has also obstructed
one of the primary functions envisaged in the original proposal,
namely the coordination of services.  Support to other NICs has
been relatively scant, the InterNIC Briefcase being one of the
few tangible products in this area, and one that was only



                                16




produced very recently.

Finally, the general approach taken in the management of the
project appears to have been excessively reactive, and evidenced
poor planning at various times.  While some aspects of the
evolution of the Internet have been extremely unpredictable, this
is not true of relatively generic business activities where the
project has experienced problems.  In particular, it should have
been possible to predict and take steps to avoid problems with
such areas as book and CD-ROM distribution, and overloading of
the 800 public-access number.

The panel's Overall Quality evaluations are provided in Table 1.


       ____________________________________________________________________
       | Cooperative Agreement Requirement | Implemented | Quality        |
       --------------------------------------------------------------------
       | Recording and tracking            | yes         |       poor     |
       | Assigning Internal Quality scores | no          |   unacceptable |
       | InterNIC Liaison Council          | no          |   unacceptable |
       --------------------------------------------------------------------

                   Table 1: Overall IS Quality Evaluation



5.3   Service Evaluation

Community Coordination

GA has made some effort to act in a coordinating role for the
other NICs, but there has been too much of an emphasis on
providing NIC services directly to end users.  Moreover, much of
the interaction with other NICs has become information sharing
rather than coordination.  Under the leadership of Kent England,
in the last six months GA has recognized this problem and started
to take action to reorient their services.  The creation of the
InterNIC Briefcase was the first step in this direction.

But the InterNIC Briefcase is only one step in the development of
documentation, which should include much needed policies and
procedures relating to network use.  The value of such
documentation and policies is obvious and remains an important
function for someone to provide.  NICs and all users would
benefit from a forum within which such policies could be
discussed, developed, and disseminated to the Internet community.

Unfortunately, however, given the highly competitive nature of
information services, the opportunity to create a web of support
organizations and shared resources has been missed.  The current
GA organization is seen as a competitor by many service groups,



                                17




and these groups see no advantage to cooperate with GA. The
opportunity to work with University NICs or specific disciplines
may still open if the benefits can be clearly outlined; however
the balance between working to strengthen the overall information
infrastructure and delivering commercial services is delicate.
Uninformed leadership at this stage could easily stifle this
coordination opportunity.

Not creating the NIC Liaison Council at the outset was a serious
departure from the cooperative agreement, and significantly
hampered the effectiveness of the InterNIC in its early stages.
Kent England's decision not to convene it during his tenure was
not made lightly but clearly speaks of GA's inability to
understand the role of this council, either as a
coordination/outreach tool or as a mechanism for leadership and
evaluation of the project and the services it offered.

The panel's Community Coordination evaluations are provided in
Table 2.  The panel's Prepared Material evaluations are provided
in Table 3.


       ____________________________________________________________________
       | Cooperative Agreement Requirement | Implemented | Quality        |
       --------------------------------------------------------------------
       | Community Coordination Services   | yes         |    poor        |
       | NIS Fest                          | yes         | unacceptable   |
       | Liaison Council                   | no          | unacceptable   |
       | International Cooperation         | unclear     | unreported     |
       | Administrative Representation     | yes         | acceptable     |
       | Educational Services              | yes         |    poor        |
       | Documentation Services            | yes         |    poor        |
       --------------------------------------------------------------------

               Table 2: Community Coordination Evaluations
                        (outreach to other NICS and organizations)



InterNIC Partner Coordination

One of the primary functions of GA within the InterNIC project
was to serve as the public communications and coordinating arm of
the enterprise.  In this capacity, GA was ultimately responsible
for the visibility of the InterNIC as a whole.  Unfortunately, GA
failed to provide visibility for its services or for its
partners' services within the R&E community, and at the same time
created inappropriate visibility within constituencies outside
the primary target audience.

GA is responsible for creating a seamless interface between the
three InterNIC partners.  An important component of InterNIC



                                18




coordination is a single point of contact for InterNIC services,
using any of the forms of electronic communication.  The ``800''
number concept failed due to the huge demand by unaffiliated
users (i.e., the general public).  GA has eliminated the 800
number and is instituting an automated telephone attendant that
provides information on reaching the other InterNIC partners.
The automated attendant service does not route calls to the other
partners, which leaves open to debate the question about whether
this service provides an adequate single point of contact.

A common directory and database service is also critical to a
unified InterNIC appearance.  The current configuration of the
database and directory service makes all of the data available
through multiple access mechanisms, but the data are neither well
organized nor easily searchable, and there is considerable
duplication of information on the servers operated by the various
partners.  A great deal of work is needed to evaluate the
usefulness of this system and to reorganize it to provide a high
quality service.  As an example of the problems, the three
awardee sites have provided dissimilar electronic mail server
interfaces -- requiring user commands in message bodies for IS
and RS, but in message subjects for DS.

Outreach and training is the area that seems to have suffered the
most neglect, and is impacting all of the service providers.
Outreach coupled with a communication plan might also be called
market research.  IS should contact customers to discover how
they use services, what their current and future needs are, and
what needs improvement.  Simply relying on unsolicited comments
is not sufficient.  The outreach activities should include
interviews, focus groups and measured uses of tools and services.
Without these activities, the InterNIC can not hope to be
responsive to the community it is charged to serve.

The panel's Partner Coordination evaluations are provided in
Table 4.


Full Service Reference Desk

The full service reference desk is a good idea for clearly
delineated communities.  GA made a good faith effort to provide
these services to the R&E community.  However, they have
recognized that too much time was spent focused on supporting the
end users.  Eliminating the 800 number was an appropriate way to
ensure that serving as NIC of first resort does not completely
swamp available resources.

The reference desk does not appear to have served the needs of



                                19




other NICs.  While a mid-course correction to focus more on the
provision of services to NICs has resulted in one tangible
product (the InterNIC Briefcase), the reference desk itself has
not been modified.  In addition, the changes that have occurred
in the Internet over the last three years call into question the
desirability or feasibility of the model of a hierarchy of NICs.

The InfoGuide provides very good online resources for the
beginner, and is starting to build a good set of resources for
the experienced user.  The InfoScout, in the person of Susan
Calcari, is the reason the InfoGuide and the Scout Report are
successful.  However, the creation of the InfoGuide for new users
creates a problem of delivering that information to new users.

Moving to entirely electronic networked information delivery
creates a hurdle for new users who are not on the network, and
may not address the needs of some users who need paper documents.
While it may be appropriate for the InterNIC to create such
materials, it is not appropriate for the InterNIC to distribute
them directly to end-users.  And given the recent explosion in
publishing on the Internet, even the creation of such documents
is again in question.

The InfoScout relies on the net-happenings mailing list for
information about current events on the Internet.  While GA has
provided listserve resources, net-happenings was started on the
personal initiative of Gleason Sackman and is not a GA project 2.
There would be severe damage to the InfoScout's ability to
continue to do its good job if they did not have this valuable
resource from which to build.

The NICLink and X window whiteboard service are also interesting
ideas that have not survived implementation.  Even so, the panel
recognizes that risks must be taken to create new services.  The
InfoScout service could have been overshadowed by an
insurmountable workload; luckily, it has not.  Good management is
the key to learning from mistakes and ensuring the balance of the
outcome leads to progress.

The panel's Full Service Reference Desk evaluations are provided
in Table 5.


Education

Training is another area where lack of focus or understanding the
needs of the target audience clearly shows.  Many of the
____________________________
   2 Sackman has no association (financial or otherwise) with the InterNIC,
   other than that the InterNIC provides the listserv software, disk space,
   and technical support for the net-happenings list.



                                20




educational seminars offered by GA have been directed at end
users rather than at the staff of other NICs.  They have entered
the commercial marketplace by offering a seminar targeted at
commercial firms.  Education is one of principal areas in which
GA has come to be perceived by many organizations as a
competitor.

The goal of providing high quality training services must start
with the identification of the target audience and their needs,
while recognizing that the commercial delivery of educational
materials has burgeoned over the last three years.  In addition,
GA has missed an important and valuable opportunity to work with
its partners to deliver training based on the Directory and
Database Services of AT&T and the Registration Services of NSI.
Both groups have clearly identified problems with their users
that could have been addressed if GA had adequately fulfilled its
coordination role.


Outreach Services

GA's outreach services suffer from the same lack of focus as
previously described in various parts of this report.  While they
continue to participate in various network and educational forums
and to produce and distribute the NSFNET Newsletter, the value of
these efforts is hard to judge, given their ``shotgun'' approach
to targeting services to audiences.

While their presentations provide brief information on all three
partners, they are barely achieving an adequate standard in
presenting information on the InterNIC partnership.  While NSI
has received more visibility than AT&T through the popular press,
the work of AT&T with directories and databases as well as their
own development work has never been presented by GA. The R&E
community largely has no idea of what AT&T is doing and thus
perceives little value from them.  This may be an added reason
why AT&T is having difficulty getting entries in their
directories.

The NSFNET Newsletter continues to showcase information about
NSFNET and associated projects, though it has not been widely
enough disseminated to have served in any meaningful way as a
communication tool beyond an already knowledgeable group.

Net-happenings has been consistently excellent.  Sackman's work
in moderating this list has been widely recognized as one of the
most useful digests of information on the Internet. 3  It should
____________________________
  3 Note: although Sackman is a member of the panel, he did not write the
         current section.



                                21




be noted that net-happenings is not a GA project, and is not a
funded activity.

The Internet Monthly report is subawarded to the University of
Southern California's Information Sciences Institute (ISI) and is
consistently delivered to the community via mailing lists and
various Web servers.

The panel's Community Outreach evaluations are provided in Table 6.



5.4   IS Self Evaluation

There is little evidence to indicate that, other than monitoring
such relatively minor items as reference desk contacts and server
accesses, GA has made significant efforts to evaluate customer
satisfaction with its services, as originally outlined in section
J of the proposal.  Informal communication with a number of NICs
suggests that they typically either ignore the InterNIC
Information Services entirely, or use them but have not had the
opportunity for significant input into their design or delivery.
It also appears that GA has not facilitated the Self-evaluation
process for other NICs in the ways that were described in section
I.3 of the proposal.



5.5   Evaluation Summary and Recommendations For Actions to the NSF

While the panel was in general agreement that GA has not provided
its agreed-upon services, the panel did not reach consensus on
recommendations to the NSF. Below we provide the majority opinion
(held by 13 of the 16 panelists) and three minority opinions
(held by one panelist each).


Majority Opinion

Given the lack of leadership and direction shown in GA's overall
management of the project (as seen in the alarming turnover of
personnel and IS's recent reorganization), and given GA's
inability to provide the services specified in their cooperative
agreement, it is the majority opinion of the panel that the
cooperative agreement with GA for the provision of IS should be
terminated as soon as possible.

Below we identify specific projects that this part of the panel
feels the NSF should continue to support, as they have proven to



                                22




be valuable to the R&E community.  We believe that the NSF should
not, however, continue to fund GA in the provision of these
services, as they are largely due to the efforts of two
individuals despite the GA management rather than because of it.

The following projects should be continued:


   o The InfoScout in the person of Susan Calcari.  In
     particular, the Scout Report, based on information provided
     by net-happenings, is extremely valuable.  Net-happenings is
     currently not funded by the NSF in any way.  This service
     should be maintained, and expanded if necessary, if the
     volume of information becomes excessive for Gleason Sackman
     to handle it alone.

   o The NSFNET Newsletter has provided a record of the
     development of the NSFNET project since its beginning.
     Currently, the paper format is expensive and therefore its
     content and distribution should be targeted.  The NSF should
     determine if this purpose continues to be valuable as they
     step out of the business of providing network services.
     While creating an electronic form of the magazine would
     allow it to be more widely distributed, there are several
     other magazines that could fill the information gap should
     this newsletter be discontinued.

   o The Internet Monthly Report is consistently delivered to the
     community electronically.  The current subaward (at the
     University of Southern California's Information Sciences
     Institute) should be retained.

   o The NIC for NICs role should be re-evaluated.  The InterNIC
     Briefcase, the project initiated by Kent England, should be
     completed and supported.  Other projects of this nature
     should be more clearly defined in terms of target audience,
     delivery, and cost, and should be evaluated relative to
     whether the commercial marketplace can deliver the same
     services.  The NSF should clearly identify the set of
     outcomes it wishes to facilitate through the development of
     NIC services within various distinct R&E community segments.

   o The information services available through InfoGuide are
     designed to appeal to new users, and serve as a significant
     source of information about the network.  These services are
     valuable to portions of the R&E community, and should be
     maintained by AT&T. The InfoScout should continue to support



                                23




     and maintain the information it provides.  The InfoGuide may
     also serve as a distribution mechanism for the NSF.

   o Whatever projects the NSF continues to support, clear
     processes for evaluation should be implemented to assure
     that they continue to be valuable to a targeted R&E
     community.


Minority Opinion #1

This panelist's overall recommendation is to continue funding GA
for those portions of the project that are currently yielding
results:  InfoGuide, InfoScout, and further development on the
InterNIC Briefcase.  The panelist believes this would require 2-3
FTEs.  The panelist thinks the remaining projects are not going
anywhere and have already been given sufficient time to prove
themselves.  The notion of the IS portion of the award appears to
be defunct.


Minority Opinion #2

This panelist feels that the GA staff provides a high level of
service in the research and development of online resources, but
continues to struggle with all other aspects of the program
requirements.  The GA cooperative agreement should be reduced in
scope to include only the following:


   o Common electronic identity.  Maintain the single point of
     contact electronic mailing address for the InterNIC, reading
     mail and forwarding as required.

   o Common telephone identity.  Maintain the single point of
     contact telephone number with an automated attendant and
     staff backup to forward calls as necessary.

   o InfoScout, InfoGuide and the InterNIC Mailing List.  This is
     a very successful service.  It should continue and build
     upon the current work, including the maintenance of current
     archives:  ``How-to'' Templates, Resource Documents,
     Historical Archive, Network Security, and publication of the
     NSF Network News on the WWW and in limited hardcopy for
     distribution through outreach activities sponsored by R&E
     organizations.

   o InterNIC Newsletter and the Internet Monthly Report.



                                24




During the next year, the NSF should organize and sponsor focus
groups of various constituencies (including commercial Internet
service providers) to determine the need for NIC coordination
activities, education and outreach to serve the R&E community.
If this research determines that these services are still
required, a clear strategy should be articulated for providing
these services is a distributed fashion with explicit
requirements for the measurement of service quality.


Minority Opinion #3

This panelist feels that, given GA's mixed record of
accomplishments as compared to their original plans; staffing
turnover throughout the project; and uncertainties related to
experience and goals of the new management, the NSF should not
commit to full funding for the remaining years of the cooperative
agreement at this time.

Assuming a qualified co-principal investigator can be hired as
InterNIC Director before the end of 1994, GA should receive some
months of additional funding (perhaps through June 1995).  A site
visit and program review should be done in the Spring of 1995.
Possibly some members of the 1994 InterNIC Review Panel could
assist the NSF in that review.

The NSF support should be reduced immediately to the support of a
more modest Information and Education program, so that activities
that are going well can continue.  (These include InfoGuide,
InfoScout, NSF Network News, and document templates, and may also
include the Reference Desk and templates for courses.)

The balance of services provided to the R&E community and to the
general public needs to be clarified, relative to the NSF
funding.

The NSF should find some other appropriate group to tackle the
InterNIC coordination and ``marketing'' roles, if those still
seem desirable.



                                25




       ____________________________________________________________________
       | Cooperative Agreement Requirement | Implemented | Quality        |
       --------------------------------------------------------------------
       | Customizable Materials            | one         |    poor        |
       |    (InterNIC Briefcase (new '94)) |             |                |
       | "How-to" Templates                | yes         | acceptable     |
       | Resource Documents                | yes         |    good        |
       | Historical Archive                | yes         | acceptable     |
       | Self Evaluation Guidelines        | no          | unacceptable   |
       | Procedures and Policy Manual      | no          | acceptable     |
       |    (being done by other groups)   |             |                |
       | Other documents                   | no          | acceptable     |
       |    (being done by other groups)   |             |                |
       --------------------------------------------------------------------

            Table 3: Prepared Material Evaluations




       ____________________________________________________________________
       | Cooperative Agreement Requirement | Implemented | Quality        |
       --------------------------------------------------------------------
       | InterNIC Coordination             | partial     | unacceptable   |
       | InterNIC Transition               | yes         | acceptable     |
       | Development of the unified        | yes         |    poor        |
       |     InterNIC interface            |             |                |
       | Common electronic identity        | yes         | acceptable     |
       | Common telephone identity         | yes         | acceptable     |
       | Common dir/dbase identity         | yes         |    poor        |
       | Outreach coordination             | yes         |    poor        |
       | Training coordination             | partial     | unacceptable   |
       | Common Trouble Ticket System      | no          | unacceptable   |
       | Communication Plan                | no          | unacceptable   |
       | NIC Locator (new '94)             | yes         |    poor        |
       --------------------------------------------------------------------

       Table 4: InterNIC Partner Coordination Evaluations



                                  26





       _____________________________________________________________________
       | Cooperative Agreement Requirement   | Implemented | Quality
|
       ---------------------------------------------------------------------
-
       | Full Service Reference Desk         | yes         |    poor
|
       |      800# (ended '94)               |             | unacceptable
|
       |      regular # + automatic referral |             | acceptable
|
       | InfoSource/InfoGuide                | yes         |    good
|
       | InfoScout                           | yes         | excellent
|
       |      Scout Report (new '94)         | yes         | excellent
|
       | NICLink                             | yes         | poor execution
|
       | Discipline specific info packets    | no          | unacceptable
|
       | Internet User Guide                 | unclear     | unacceptable
|
       | InterNIC Mailing List               | yes         |    poor
|
       | NetHappenings                       | non-GA      | excellent
|
       | Smart Card                          | no          | unacceptable
|
       | X Windows whiteboard service        | no          | unacceptable
|
       | Tracking queries                    | yes|        |    poor
|
       | Common trouble ticket System        | no          | unacceptable
|
       | Quarterly Reports                   | yes         |    poor
|
       ---------------------------------------------------------------------
-

          Table 5: Full Service Reference Desk Evaluations



    ______________________________________________________________________
    | Cooperative Agreement Requirement     | Implemented | Quality      |
    ----------------------------------------------------------------------
    | Outreach Services                     | yes         | acceptable   |
    | Training Seminars                     | yes         |    poor      |
    | NSFNET Newsletter                     | yes         |    good      |
    | Internet Monthly Report               | yes         |    good      |
    | Network Security                      | no          | unacceptable |
    | Increasing Network Involvement (K-12) | unclear     | unreported   |
    | Technology in the Classroom           | unclear     | unreported   |
    ----------------------------------------------------------------------

            Table 6: Community Outreach Evaluations



                               27




6   InterNIC Registration Services - Network Solutions, Inc.


6.1   Service Overview

The functions included under Registration Services include:


   o Overall administration of the Internet Protocol version 4
     (IPv4) address space.  NSI also coordinates with Reseaux IP
     Europeenne (RIPE) and the Asia Pacific Network Information
     Center (APNIC) in this task.  NSI is responsible for
     delegating the address space to all first level NICs.  NSI
     also operates and manages root domain name servers.

   o Registration and administration of U.S. IPv4 addresses.  NSI
     is also the NIC of last resort for registration and
     administration of addresses and names for those parts of the
     Internet that do not have their own registration authority.

   o Administration and assignment of the .GOV, .NET, .COM, and
     .EDU Internet domain names.  This task includes the
     operation of primary domain name servers for these name
     spaces.

   o Registration and continued administration of the legacy
     WHOIS database.

   o Through a subaward to ISI (Jon Postel as principal), the RS
     administers the .US Internet domain name space.

   o Development of tools for performing InterNIC RS functions,
     such as the Referral WHOIS (RWHOIS) effort.



6.2   Evaluation and Recommendations

The panel congratulates Network Solutions Inc.  (NSI) for their
excellent work on providing registration services to the Internet
community.  The RS portion of the InterNIC project has responded
well to the demands of the rapid growth of the Internet.  NSI has
directly facilitated Internet growth by effective allocation and
administration of Internet names and addresses.

NSI has responded to the exponential growth of the Internet by
applying new approaches in the performance of their RS duties.
In their role of overall administration of the IP name and
address spaces, NSI has been responsible for the delegation of



                                28




management of the IPv4 address space and country-based name
spaces (e.g., .FR, .UK, .JP, etc.).  During NSI's tenure in
operating RS functions, the Internet has shifted from centralized
management of these functions to a distributed model of
management and administration.  This transition has gone very
well, thanks in part to NSI and in part to the organizations that
have stepped up to build this Internet infrastructure.

The WHOIS database of network administrators has gotten to the
point that a single centralized flat database is inappropriate.
NSI has developed RWHOIS as a simple protocol to facilitate
building a distributed database in place of the original WHOIS.
It appears that the network operations community is picking up
RWHOIS until a more general solution to Internet white pages
becomes available.

The panel observes that NSI will not be able to keep up with the
overall load generated by the continued exponential growth of the
Internet.  NSI is encouraged to look for additional avenues for
applying technology to solving some of their growth problems.
Many of the tasks NSI performs currently have work steps that
require human processing.  The panel believes that NSI should
continue to press to make these steps more dependent on computer
processes if at all possible.  The panel also recommends that the
NSF support those efforts, and also be prepared to allocate more
resources to NSI to help them meet their increasing work load.

It is clear from the materials presented by NSI that a primary
culprit in the RS work load is the .COM domain.  The current
management of .COM is not scalable since .COM is a flat domain
name space, and thus the load of administering .COM falls solely
on NSI. At present, the management of .COM is paid for by the
NSF, and hence increasing demand for .COM registrations will
require increasing support from the NSF. The panel recommends
that NSI begin charging for .COM domain name registrations, and
later charge for name registrations in all domains.  Over the
long run, the panel recommends that NSI charge for all IP
registration services.

During panel discussions a consensus emerged on a possible
charging model that requests an initial fee for registering a
name, and a recurring annual fee for subsequent administration of
the name space, to cover costs due to updating entries, ensuring
uniqueness of names in the name space, operation of root name
servers, etc.  It was noted that the charge for initial
registration and recurring fees did not logically have to be the
same amount.  However, charging the same amount would allow NSI
to state that everyone, including those who already have Internet



                                29




domain names, will have to pay the same amount within the initial
12 months and this might prevent a last minute, ``get them while
they are free'' rush on domain names.  NSI should consult with
the NSF in the development of such a policy.  The NSF needs to
plan mechanisms for defraying the costs for institutions, such as
U.S. R&E sites that would fall under the .EDU domain, that may
not be able to bear the new charges directly.  In the ideal
scenario, any new plans should be in the direction of a fee to
the end user of a name, and thus would facilitate the NSF's
getting out of the name registration business.

During the review, NSI discussed their encounters with the legal
issues that have arisen in their management of the .COM domain.
It appears that companies are considering pressing their trade
and service mark claims into registration of similar names in the
.COM domain.  The panel was disturbed to hear that the NSF and/or
the U.S. government does not provide a legal umbrella under which
NSI can manage the Internet domain name space.  The panel
encourages the NSF to support NSI on legal and policy-related
issues that stem from management of the domain name space, and
also recommends that NSI and the NSF work to place the operation
of the domain name system into a framework that provides suitable
legal protection of the registration function.  NSI should work
in a leadership role in the development of such a policy.

The panel recognizes NSI's success in fostering the
internationalization of the RS functions to date.  Many of the
policy directions the panel is recommending, such as fee for
service for name registration and an operational legal framework,
will require an international framework if these policies are to
be successful.  NSI is encouraged to work within the Internet
community to foster the development of the necessary
international frameworks.

The panel would also like to congratulate Jon Postel and ISI on
their work in developing the administration of the .US domain as
a subaward under the RS InterNIC award.  In particular, ISI has
developed informational materials on the .US domain that will
help the future deployment of zones within the .US domain.  The
panel encourages ISI to continue to work on scaling the .US
domain so that the U.S. Internet community does not have to
revisit the problems exemplified by the current morass in the
.COM domain.



                                30




7   NSF Meta-Questions


As part of the review process, the NSF provided the panel a set
of ``meta-questions'' intended to solicit recommendations about
the viability of aspects of the Network Information Services
Manager concept.  Below we provide an overview, and responses to
each of the meta-questions.



7.1   Overview

The NSF has been incredibly successful in its work to develop the
physical network infrastructure by providing funding and guidance
at multiple levels, from the end user R&E institutions
(connection grants) to mid-level and national backbone services.
The exponential growth of the Internet can be directly attributed
to the assistance given in its development through these NSF
programs.  A combination of small grants and larger cooperative
agreements has allowed network growth in response to rapid
changes in technologies and the user base.

Because of the degree of community input during the solicitation
process and the NSF's previous success with the physical network,
there were high expectations throughout the R&E community that
the InterNIC would serve as a similar NSF-funded focal point to
jump-start the support and information services required by the
Internet's education and research users.  However, the
expectation that the InterNIC would provide a focus for a similar
high level of development and coordination for Information
Services has not materialized.  In its review of this project,
the panel recognized that a number of issues remain before a
cohesive network services program will develop.

The InterNIC awards were the first Collaborative Cooperative
Agreements requiring significant self-coordination among a team
of awardees and the presentation of a ``seamless interface'' to
the user community.  In addition, there was the expectation that
other NICs, particularly interdisciplinary NICs, would be
developed that would draw on the InterNIC as a resource.

As has been indicated in the review of the GA component (see
Section 5), the failure by the organization with the lead
responsibilities for coordination and outreach has had a negative
impact on the better developed programs by the other partners.
Further, it negatively impacted any potential for fostering
development of other NIC programs.  Instead of a cooperative
effort that could draw on and focus the many projects already
underway in the community, GA chose a predominantly solitary



                                31




course that did not create the vital interorganizational efforts
that might have been possible.  At the same time, there was
little acknowledgment of the massive changes that were occurring
in the Internet and the need for rapid redirection to meet these
new challenges.



7.2   NSF Management of the Cooperative Agreements

Given the problems that developed with this program, some
consideration needs to be given to the role of the National
Science Foundation in its work with the project.

The advantage of a cooperative agreement is that it provides
program flexibility in the face of changing circumstances.
Through such agreements, the NSF can have continued input into
the development of the program structure, and provide the
participants of any single program with the knowledge that the
NSF's staff has through work with multiple network projects.  It
can also ease the problems of rapid program change in the face of
often slow government solicitation and award processes.

The panel offers three suggestions in this context.

First, the NSF should critically consider the viability of future
cooperative agreements (and future proposal review panels'
recommendations about these agreements) in light of the InterNIC
experience.  Perhaps it is simply not viable to expect
significant self-coordination among a team of awardees.

Second, the NSF should carefully review its capabilities for
managing major cooperative agreements.  Management theory is
clear about the commitment required for maintaining
multi-organiza-tional relationships.  Because of the multiple
agreements involved, the NSF took on a greater responsibility for
that management and does not seem to have had adequate resources
to fully assist in problem resolution, despite what seem to have
been excellent efforts by NSF staff.  While this does not
necessarily indicate that such multiple awards are prone to fail,
it suggests that NSF/DNCRI must carefully consider what its focus
and management resources are, especially in the face of the
pressures to continue in its physical infrastructure support
projects.  As major initiatives are started in new areas such as
Information Services, careful consideration should be given to
the needs for staff monitoring of such major projects.

Third, the panel recommends that the NSF make more extensive use
of outside experts throughout the life of each cooperative
agreement.  One of the outstanding achievements for the NSFNET



                                32




transition has been the external expertise brought about through
hiring ``consultants'' such as Peter Ford to assist the NSF staff
by bringing direct experience from the field. 4  A similar
consulting group for Information Services would have provided
valuable insights, and assured that critical issues were raised
as the InterNIC took shape.  Because the solicitation process
moved slowly, such consultants would have provided additional
insight into changing conditions as the awards were made.  If the
changing environment had been addressed early in the process,
some problems (such as failure to focus on NICs rather than
end-users) might have been flagged earlier and possibly resolved.
The consultant model may be a direction for the NSF to pursue as
it further studies what should be done for creating a cohesive
Network Information Services framework.



7.3   Requirements for Ongoing "Customer" Feedback

The InterNIC project failed to develop specific methods for user
feedback other than unsolicited individual user electronic mail,
which the panel feels is not an adequate means of evaluation.
Any major project such as the InterNIC that is required to serve
the community should have some formal process for community
input.  The NSF may wish to consider such as a requirement in
future solicitations.  For example, an external advisory board
that had a solid sampling of ``customers'' could have assured
issues were raised more speedily about changing environments
without necessarily waiting two years for a review panel to be
summoned.  When millions of dollars are involved, the NSF needs
ongoing methods to provide evaluation and ensure that the
community that is supposed to be served does indeed receive the
expected services.  Given the flexibility possible in a
cooperative agreement, such external advice to the NSF and
project management can be of great value in providing ongoing
corrections as well as ensuring that projects retain direct
contact with ``customers.''  Such an external advisory board
could also benefit the remaining years of the InterNIC project
and should be considered.

____________________________
  4 Note: although Ford is a member of the panel, he did not write the
         current section.



                                33




7.4   Appropriateness of Informal Relationship between the InterNIC and
CNIDR

The Clearinghouse for Networked Information Discovery and
Retrieval (CNIDR) has been funded by the NSF under a separate
cooperative agreement from that of the InterNIC, in response to
an unsolicited proposal.  Because the NSF felt CNIDR's activities
were of complementary value to the InterNIC, they initiated an
informal relationship between CNIDR and the InterNIC. While a
formal review of CNIDR would not be appropriate for this report,
the NSF requested that the panel examine the relationship between
CNIDR and the InterNIC.

In general, the panel agreed that the liaison between the
organizations has been beneficial, particularly with the
Directory and Database Services (AT&T) where information tools
are a key to service success.

Based on the information available at the review, the panel
recommends continuation of the informal collaborative
relationship, which seems to be working well.  Given the problems
identified with coordination among the InterNIC organizations
primarily attributable to GA's poor performance, it does not seem
advisable to complicate organizational relationships with formal
inclusion of CNIDR in the InterNIC at this time.



7.5   The Viability of the NIS Manager Concept

The solicitation that resulted in the InterNIC awards was
prepared nearly three years ago.  Since that time, many changes
have occurred (see Section 3).  While the panel focused on
evaluating performance based on submitted proposals, the issue of
the InterNIC's overall effectiveness in establishing a network
information services framework also arose.  It was agreed that in
general, this question could not be fully addressed because the
primary problem of GA's failure to perform raised unanswerable
questions about what might have been.  There was little question
from the panel's discussions, which represented voices from a
broad sampling of the user community, that the needs identified
in the solicitation still exist.

While no consensus was reached on the issue of what changes
should be implemented, there was agreement that no single program
could meet the full spectrum of needs for Network Information
Services.  While the panel is in full agreement that the
successful InterNIC services were important and should be
continued, there was a sense that the current efforts are



                                34




insufficient.  In general, the panel sensed that the InterNIC
program may need to be supplemented with parallel projects if the
goals are to be achieved, both to fill gaps not clearly ``owned''
by any of the participants, and to implement projects that have
failed to materialize as expected from GA.

While the discussion on this was brief, the panel recommends that
the NSF reexamine the Information Services program to make it
more comprehensive.  An example was given by one panelist, who
noted that physical infrastructure success has been the result of
seeding at both the top (backbone) and bottom (stub network)
levels, encouraging a community commitment to coordinated
development.  The panelist indicated that there was no similar
multi-pronged assault on Information Services.  A panel subgroup
came up with a similar suggestion aimed at encouraging a
complimentary program to assist other NICs in developing projects
that would be widely disseminated to spread the base for NIC
programs.  In such a program, the NSF could encourage projects
that were cooperative and broadly beneficial -- a contrast to the
conflict between commercialization/self-support and community
service that continues to be a difficult issue for the InterNIC
participants to resolve.  Through such an enhanced program, the
remaining InterNIC services could be better leveraged and built
within a community framework that would enhance the services that
are already developing.  The capability to encourage
collaboration and cooperation also builds on strengths within the
NSF that can be helpful in providing focus for the larger R&E
community.

The panel also suggests that the NSF consider how to reinvigorate
this area so that ``the next time'' it will work.  A more
targeted approach may be appropriate, focusing either on
communities of interest (e.g., K-12, or K-12 in Illinois, etc.)
or on professional communities (professional societies, chemists,
historical societies, etc.).  The panel suggests care in looking
at what InterNIC activities are heavily used today, and suggests
trying to learn methods and overall strategies from them before
issuing any new solicitations.  The panel recommends planning
grants to stimulate workshops and program/partnership
development, and also suggests looking for more industrial
partnerships (e.g., with Dow Jones, AOL education, Newspapers,
CNN, Knight Ridder labs, etc.)  Finally, some members of the
panel felt that 5 year awards are a mistake, and that the NSF
should consider shorter time horizons, such as 2 year awards.



                                35




     A    InterNIC DS's Responses to Panel Questions


     The questions below were answered by Erik Grimmelmann, PI of the
     DS portion of the InterNIC.


Question  The sense of panel is that the directory of directories is
          an extremely important service but is not used at present
          because it is not sufficiently populated.  What are your
          numeric targets for populating this directory, and how do
          you expect to reach those targets?

Response  We agree with the committee that the directory of
          directories is extremely important and needs significant
          expansion.

          The Internet community is extremely large, and a popular
          resource can easily be swamped.  Many resource owners cannot
          support large scale demand for their resources, so we
          believe we should list resources only when we have the
          ``owners'' permission.  On the assumption that this policy
          remains in place, our targets are 1,000 entries by the end
          of contract year 2, 3,000 by the end of year 3, 6,000 by the
          end of year 4, and 10,000 by the end of year 5.

          We expect to reach these numbers by devoting a person to
          this process full time.  This person will prepare
          descriptions of resources and submit them to the owner for
          approval.  We will increase outreach activities, and ask
          third parties to submit information on resources they feel
          we should list (via electronic mail and via an HTTP form
          interface).  Resource owners could also use the form
          interface to submit entries to the directory of directories.

Question  Given the work you do with directory services, do you
          believe it's appropriate that you provide database services
          to the community?  Does it have an impact on your ability to
          deliver directory services?  How do you prioritize/balance
          these demands?

Response  We believe that our support of directory services is
          significantly more important than our support of database
          services, and our resource allocations reflect this (only
          about 5% of our resources are dedicated to database
          support).



                                     36




          However, a number of the databases we hold (RFCs, IETF
          documents, etc.)  are quite useful to the R&E community,
          especially since we WAIS index these documents.  There is
          also synergy between the tools we need to support databases
          and the tools we need to support the directory of
          directories (WAIS, Gopher, WWW, mail server, and support
          utilities).  We expect the current split in resource
          allocation will continue.

Question  If database services are deemed appropriate, what steps will
          you take to determine what the R&E community needs in terms
          of databases?

Response  Databases we handle for a fee help cover costs of running
          the InterNIC. In these cases the content owner determines
          the target audience.

          Databases that we support without a fee will be focused on
          communities such as K-12, libraries, and other affinity
          groups that primarily serve the R&E community.  We will
          continue to seek guidance from the NSF on case by case basis
          for additional no-fee databases.

Question  How can you make your services more appealing to the
          community?  Packaging, organization, outreach, ...

Response  While we will continue to refine our packaging and
          organization, we feel the greatest benefit will come from
          increased outreach.  We will increase attendance at
          conferences in target communities and aggressively solicit
          and follow up leads.

Question  After your 2 years of experience, do you believe you are
          being required to deploy services that depend on solving
          some as yet unsolved research problems?  For example,
          technical problems of providing a directory of directories,
          or privacy problems of white pages services?

Response  Yes.

          There is a demand for a large scale white pages service that
          cannot be fully met with current technology.  RFC 1588
          recognizes that the Internet needs systems that efficiently
          tie together a variety of directories using different tools.
          This is not yet a solved problem.

          The privacy dimension also presents issues in terms of the
          willingness of many organizations, especially corporations,



                                     37




          to make their directory information available.  Directory
          systems must be made more secure.

          Accuracy will also remain a problem as long as directories
          are not tied to systems that are updated as part of normal
          business.  Corporate directories are typically tied to
          payroll/personnel databases; published telephone directories
          are tied to telephone customers who pay a bill every month
          and thus validate their address.  We need to find ways to
          tie our directory services into systems that already have
          accurate copies of directory data.

Question  The panel felt you provided a lot of data, but did not
          analyze/digest these data sufficiently to permit independent
          evaluation.  What will you do to provide meaningful customer
          feedback analysis?

Response  We agree that some additional analysis could be useful, but
          much of the data is anecdotal and the commenting population
          is self-selected.  We will explore inexpensive mechanisms to
          analyze the data we have.  We have an REU student who has
          started to analyze the data, but it is quite diffuse and
          does not yet provide useful information for planning.

          We will also explore mechanisms to inexpensively survey the
          target community and help direct our outreach efforts.

Question  What do you see as your strategic directions through the end
          of the award?

Response  The Internet is changing so rapidly that a tactical focus is
          more important than a strategic focus.  However, we do
          expect to continue our focus on service, and on providing
          high quality, reliable, accessible, and accurate services.

          For the directory of directories, we intend to follow the
          plan described in our answer to question 1 to increase the
          number of entries and thus the overall utility of the
          collection.  As noted in our answer to question 6, we will
          also explore mechanisms to survey the target community and
          discover what subject areas of the directory of directories
          should be the focus of our efforts to increase the number of
          listings.

          For directory services, we will continue to work towards the
          Internet directory envisioned by RFC 1588.  We will provide
          a testbed for real-world implementations of tools that
          provide gateways between different sorts of directories, and



                                     38




          will continue to work with the IETF and the research
          community on the design and architecture of such tools.

          As noted in our answer to question 2, we believe that
          databases are less important strategically than directory
          services, but we will continue to support databases that are
          of interest to the R&E community or which provide added
          value for the InterNIC.

Question  What are you doing to coordinate with your international
          counterparts?

Response  Our white pages effort includes participation in the
          world-wide PARADISE project.

          In general, we have interpreted our charter to target the
          U.S. R&E community.  We have certainly not excluded overseas
          resources from the directory of directories, and we have
          begun to make copies of RARE RFCs available on our server
          under an agreement with RARE.

Question  What is the service you're providing that the U.S. R&E
          community can't do without?

Response  Our bringing together disparate services to provide the
          directory of directories and our directory services.  We
          also provide a focus and a real world testbed for emerging
          technologies in information access and directory
          integration.

          In addition helped to revitalize the X.500 pilot in the
          U.S..

Question  What are your opinions on the questions you asked the panel?
          What did you learn over the last 2 years?  Suggestions?

Response  Issue 1:  Resource owner must approve listing and
          description for inclusion in Directory of Directories.

          As noted in our answer to question 1, we believe we should
          continue the policy of requiring owner approval to list a
          resource.

          Issue 2:  How to focus our services on the Internet R&E
          community

          As we focus our outreach activities on the R&E community,
          the center of gravity of our services will naturally tend to
          follow the needs of the community.  In addition, some of the
          market research described in our answer to question 6 could



                                     39




          help us identify segments of the R&E community that we are
          not targeting adequately.

          Issue 3:  Growth in volume:  DS servers are potentially
          default targets of commercial services.

          Tony Hearn's analysis which he mentioned in the meeting
          indicates that AOL is a good target for K-12 teachers who
          are an important part of the R&E community.  This is
          consistent with with anecdotal evidence we had heard
          previously.  Given this input, no action seems to be
          necessary at present.

          Issue 4:  Resource allocation:  Do we aim for high
          availability for all services, or split servers so high
          usage applications do not hurt performance for all users?

          We believe that we should continue our emphasis on high
          availability.  This is consistent with our intent to provide
          high quality service.

          We will investigate reconfiguring our systems so that
          different services target different servers as primary, but
          all servers will continue to offer the complete suite of
          applications.

          Issue 5:  Should we put more emphasis on ``how to'' kits and
          on-line help and tutorials for directory applications?

          Yes.  Where this involves significant effort, we will seek
          guidance from the NSF regarding the priority of such efforts
          and the likely impact on the community.



                                    40





     B    InterNIC IS's Responses to Panel Questions


     The first question below was directed to Karsten Blue, the
     president of the newly reorganized CERFNet.  All other questions
     were answered by Kent England, departing manager of the IS
     portion of the InterNIC.


Question  Karsten Blue:  what is your strategic intent for the
          InterNIC?

Response  Our strategic intent for the InterNIC is to serve the
          Internet community by making it easier to use the Internet.

Question  The panel feels your activities lack focus, specifically for
          supporting the R&E community, and that you haven't achieved
          sufficient visibility in that community.  Please tell us how
          you will change to solve the problems of the R&E community,
          and to achieve visibility in that community.

Response  It is true that the changing constituency of the Internet
          has caused us problems in reaching our target U.S. R&E
          audience.  We have worked hard this year to re-orient our
          approach to finding the right targets in the ever widening
          Internet audience, by restricting uncontrolled access to our
          human resources by the general public through the open
          access Reference Desk (but not restricted from online
          resources, like InfoGuide).

          Our U.S. R&E audience consists of both end users and other
          NICs.  Our objective is to support the needs of the R&E end
          user community directly primarily with online information
          resources and by supporting other NICs, which support R&E
          end users directly, through support for the basic functions
          of a NIC.

          The most effective way we can support R&E end users directly
          is through our online information resources, such as
          InfoGuide, Scout Report, NSF Network News, and
          net-happenings.  The other effective way to support R&E
          users is through support to other NICs.  I chose to tackle
          the issue of online information resources during the first
          quarter of this year by introducing the InfoGuide HTML
          resource, the NSF Network News bi-monthly (hardcopy and
          online), and the weekly Scout Report.  I chose to tackle the
          issue of NIC support in the second and third quarters of



                                     41




          this year through the introduction of USV-Web, NIC Locator,
          and the InterNIC Briefcase and our interactions with FARNET,
          SIGUCCS, and Educom.

          I feel that these efforts are effectively focussed on the
          R&E community that is supported by our NSF funds.  I feel
          that the combination of online resources and support to
          other NICs is our most effective means to serve the needs of
          diverse users within the U.S. R&E community.

          With respect to visibility, if I can assume that InterNIC
          Information Services needs to become more visible as CNIDR
          has achieved visibility, then I would say that the InterNIC
          Manager needs to focus more attention on coordination within
          the community, by attending more conferences and workshops
          and putting forward the InterNIC case and our service
          profile.  I believe that the InterNIC ``Handout'', included
          in the black folder, is the story we have to tell about our
          services today.

          Another way we can achieve visibility is through more
          intensive support for specific pilot or demo programs.  For
          example, we must work with others to create more
          discipline-specific NICs, such as the Agriculture Network
          Information Center (AGNIC). Another hypothetical example
          would be training and Help Desk support for a specific K-12
          pilot program with the objective of demonstrating how a
          school system would develop user support for Internet
          access.

          The online R&E community can be reached by the InterNIC
          staff by more active participation in newsgroups and mailing
          lists and by offering pointers to the InfoGuide and other
          InterNIC services, when appropriate.  For example, many
          questions asked time and again by beginners can be answered
          by browsing the InfoGuide or doing a search on the contents
          of the InfoGuide and the net-happenings archive.  A directed
          effort to have a constant presence on the network would do
          much to spread the word on InterNIC services.

Question  The solicitation and your proposal response call for a NIC
          of NICs focus.  The panel feels you do not meet this
          requirement.  Is this a mistaken perception, and if so how
          are you meeting this need?  In particular, what will you do
          to establish a NIC liasion council?

Response  The issue of NIC support has only recently been addressed,
          after the introduction of our online information resources.



                                     42




          The Briefcase and the Whole Internet seminar are the first
          two concrete InterNIC services for other NICs that we have
          recently introduced.  Our current suite of services is
          outlined in our InterNIC ``Handout''.

          I feel that the NIC liaison council needs a focus and that
          the initial focus is provided by these two new services.  If
          the NIC liaison council charter is to advise InterNIC on NIC
          support services, then we are ready now to solicit
          participation in an annual or semi-annual convocation of the
          Liaison Council, timed to coincide with an appropriate NIC
          conference or workshop.

          I would very much appreciate feedback from the Review Panel
          about which NIC conference or workshop we should coordinate
          with.  I would suggest SIGUCCS or a CNI tie-in.  I would
          also appreciate feedback on the makeup and size of the
          Council.

Question  It appears you have tried to take on many services that
          other providers can and do handle.  How will you delegate or
          exit from efforts, and as a gov funded activity how will you
          achieve fairness to other providers/enterprises (commercial
          and otherwise) - authors, seminar presenters, service
          providers?  How will you balance this w/ need to do cost
          recovery?

Response  We originally intended to sell books by mail, as well as
          CD-ROM subscriptions.  These efforts are more effectively
          done by others and we have suspended these efforts.

          Those efforts that are wholly or substantially funded by NSF
          or other government funds should not be competitive with
          other commercially available services.  These subsidized
          efforts should be undertaken to provide services that are
          not commercially available until such time as commercial
          alternatives are available.

          Those efforts that are wholly unsubsidized may be
          competitive with other commercially available services and
          the competition will not suffer through inappropriate
          cross-subsidy.  The InterNIC seminars are an example of
          this.  There is no reason why InterNIC can't offer seminars
          in competition with others when there are no subsidies
          involved.

          It is in the area of cost recovery from partially subsidized
          activities, such as the Help Desk, where the most difficulty



                                     43




          is present.  I suggest that we attempt to maintain a
          separation between the subsidized efforts, such as the Scout
          Report, and the cost-recovered efforts such as the seminars,
          and avoid mixing of subsidized and unsubsidized activities.

Question  When you changed managers before, you lost a lot of
          momentum.  How will you maintain momentum this time?

Response  General Atomics must not delay in recruiting a new Principal
          Investigator willing to accept responsibility for the
          InterNIC. GA must coordinate with NSF and the Review Panel
          to find a PI that is acceptable to all parties.  I feel that
          GA has a limited time to accomplish this transition or else
          the momentum will again be lost.

Question  How will you provide a seamless service among the 3 InterNIC
          entities?

Response  Seamless service is achieved first by close cooperation
          among the partners and second by effective presentation to
          the clients.

          The four partners of InterNIC (General Atomics, AT&T, NSI
          and MCNC) are becoming an increasingly more coordinated team
          as we learn to work together.  This has led to closer
          cooperation on a number of issues, particularly our support
          for RS.

          The seamless interface to our constituents is not simply a
          matter of a single phone number or a single point of
          contact.  It is through the coordinated outreach, for which
          GA is primarily responsible, that we maintain a coherent
          front.  As we achieve more visibility we will be recognized
          as the four constituents of one InterNIC service.

Question  What will you do to get real customer feedback, statistics,
          etc.  that allow independent evaluation?

Response  Hard data comes from our online servers with statistics
          about information served via WWW, Gopher, FTP, electronic
          mail, etc.  We have published statistics in our monthly
          reports.  Please comment specifically on what you would like
          to see in our regular statistical reports that we do not
          currently provide.

          We also retain customer feedback in the form of kudos and
          comments.  These comments have been reported to NSF and our
          partners in our quarterly review slide presentations which I



                                     44




          decided, for reasons of space, not to include in your
          binder.  This has resulted in the oversight of not providing
          you with specific instances of this kind of qualitative
          feedback which we can rectify by sending you some of our
          presentation data.

          A way to get more objective evaluation is to conduct market
          research by asking a statistical sampling of our clients how
          we are doing.  I think this would be premature at this point
          and expensive at any rate.

          Frankly, when InterNIC IS is sufficiently ``visible'' to the
          community, it will be sufficiently visible to you and the
          question of evaluation will be answered by you and by others
          in the community commenting in public media, such as on
          mailing lists.

Question  What is the service you're providing that the U.S. R&E
          community can't do without?

Response  All of the services that InterNIC IS provides are
          competitive in the sense that others can, will and do
          provide comparable services.  We are not a monopoly
          franchise, like InterNIC RS.

          The most important thing that InterNIC IS can provide to the
          U.S. R&E community are certain value-added services that
          help organize the Information Reservoirs and Information
          Streams that we are in the process of building right now.

          NSF has been remarkably effective supporting network
          connectivity infrastructure.  This is no longer necessary as
          the Internet has evolved to the present.  We are in the
          early stages of building a real information infrastructure
          and we don't yet know how to provide the day-in day-out
          production services to manage this information
          infrastructure.

          You should not let the current start-up, bootstrapping, and
          management woes of InterNIC IS distract you from the real
          need of the U.S. R&E community for support of the
          information infrastructure.  Please consider the evolving
          roles I outlined at the end of my presentation and then
          advise on how NSF may play a vital role in the evolution of
          the information infrastructure comparable to the role DNCRI
          played in the formation of the Internet connectivity
          infrastructure in the period from 1987 to 1994.



                                     45




          I firmly believe that NSF has a role in supporting this
          information infrastructure by providing structure for the
          reservoirs of information (such as InfoGuide) and editorial
          content on the streams of information (such as the Scout
          Report).  The information architects and system mechanics
          are very important parts of the production services that
          InterNIC must invent in the future.



                                     46




     C    InterNIC RS's Responses to Panel Questions


     The questions below were answered by Scott Williamson, PI of the
     RS portion of the InterNIC.


Question  What solutions does NSI see for their legal and scaling
          problems?

Response  NSI sees the need for an International non-profit
          organization to accept the responsibility for the liability
          associated with registrations.

          We see the need for strong policy/procedures for
          registration.  The organization that has the liability must
          adopt the policy/ procedures for Internet registrations.

          Further, we feel that the Internet Society's treatment of
          the standards process should serve as a model for the
          registration process.

Response  NSI sees the need for our scaling problems as follows:

          Distribute both the data through RWHOIS and the registration
          allocation process through regional registries.  Also,
          potentially establish third (3) level regional registries.

Question  How soon could you implement a charging scheme?  Do you have
          a plan in place to handle billing?

Response  NSI could implement a charging scheme in two weeks starting
          with the acceptance of checks.  Followed by a credit card
          charging scheme.  To implement a credit card charging scheme
          NSI will exercise the first option of upgrading an existing
          account which would take from one (1) week to one (1) month
          to execute.  Otherwise, if a charging scheme must be
          initiated from scratch, research tells us it will take from
          one (1) to four (4) months to implement.  This scheme will
          include how to handle the billing.  The charging scheme
          would apply to .COM, .ORG, and .NET.

          For existing records write software that will filter through
          the database and send a notice to the established POC via
          e-mail within their activation month.  NSI recommends that
          this would begin three (3) months after activation of the
          charging system.



                                     47




Question  What is your plan for spinning up the 10,000+ DNS servers we
          will need in the U.S. 3 years from now, and how will this be
          supported in the future?  Will the current volunteer scheme
          be able to meet the exponential growth?

Response  We expect the name servers to be volunteered by
          participating communities.  They will initiate their
          incorporation into the domain system.

Response  NSI believes the volunteer scheme will be able to meet the
          exponential growth through additional levels of delegation.

Question  Upon which of the services offered by the other InterNIC
          partners are you dependent?  What
          increases/decreases/changes would impact your activities?

Response  NSI is dependent on the HELP DESK services provided by
          Information Services (IS) to handle the basic Internet
          questions from the user community.  If this service did not
          exists NSI would obviously experience a considerable
          increase in what is already a high volume of ``What is the
          Internet'' type questions.

Response  Better education of the Network Service Providers (NSPs) by
          IS would result in a decrease of inquiries via phone calls
          to NSI.

Question  How will NSI handle explosive growth in non-U.S. sites?
          What will you do to further the globalization of management
          of NSI? (Note:  This question originally asked about both
          NSI and the Internet Assigned Numbers Authority (IANA), but
          it was pointed out that the IANA role is not funded under
          the InterNIC awards and hence that this question should be
          limited in scope to the NSI.)

Response  Domain Names in other countries are primarily under country
          codes and those are delegated to administrators in each
          country.  Currently the number of non-U.S. sites registering
          in .COM is very small.

Response  Address Numbers in non-U.S. sites are primarily handled by
          regional registries.  Additional registries may be
          established to handle the growth.

          Continue to strengthen the ties between registration and the
          Internet Society, which is an international organization.



                                     48




          Discuss registration policies in International technical
          committees such as in the IEPG (Internet Engineering
          Planning Group).

Question  The panel perceives a problem with international data being
          applied to the global information database in a timely
          manner.  What will you do to remedy this problem?

Response  International updates go into the same queue as all other
          requests.  We are working out automatic updates from
          regional registries, first with SWIP (Shared WhoIs Project),
          and later with complete delegation via RWhois.

Question  Do you foresee any situations that might arise in the future
          that could cause NSI to be unable to continue functioning in
          registration services?

Response  Litigation against NSI would effect NSI's ability to do this
          job.  Any other changes to NSI should not effect NSI's
          commitment to continue this project.

Question  Beyond the obvious, which of the services you offer do you
          feel the U.S. R&E community cannot do without?

Response  It is clear that the R&E funding should not support the
          growth of the .COM domain, yet there are important elements
          of the .COM which are an active part of the R&E community.



                                     49




D    List of Acronyms


 AGNIC           Agriculture Network Information Center (AGNIC)
 AOL             America On Line (information service provider)
 APNIC           Asia Pacific Network Information Center
 AT&T            American Telephone and Telegraph Corporation
 CD-ROM          Compact Disc Read-Only Memory
 CERFNET         California Education and Research Federation Network
                 (Owned and operated by General Atomics)
 CERN            Conseil Europeen pour la Recherche Nucleaire
                 (European Center for Nuclear Research)
 CNI             Coalition for Networked Information
 CNIDR           Clearinghouse for Networked Information Discovery and
                 Retrieval
 CNN             Cable News Network
 .COM            Commercial institution (Domain naming branch)
 CU-SeeMe        Video Conferencing Tool developed at Cornell University
 DNCRI           Division of Networking and Communications Research and
                 Infrastructure (NSF)
 DNS             Domain Naming System
 DOE             U.S. Department of Energy
 DS              Directory Services (AT&T portion of InterNIC)
 .EDU            Educational institution (Domain naming branch)
 EUNET           European UNIX Network
 FARNET          Federation of American Research Networks
 .FR             France (Domain naming branch)
 FTE             Full-Time Equivalent
 FTP             File Transfer Protocol
 GA              General Atomics Corporation
 .GOV            Governmental institution (Domain naming branch)
 HTML            Hypertext Mark-up Language (WWW)
 HTTP            Hypertext Transfer protocol (WWW)
 IANA            Internet Assigned Numbers Authority (currently operated by
ISI)
 IEPG            Internet Engineering Planning Group
 IETF            Internet Engineering Task Force
 IP              Internet Protocol
 IPv4            Internet Protocol, version 4
 IS              Information Services (GA portion of InterNIC)
 ISI             University of Southern California's Information Sciences
                 Institute
 InterNIC        Name adopted by NIS Manager awardees for their unified
service
 .JP             Japan (Domain naming branch)
 K-12            Kindergarten through Twelfth Grade networking
 MCNC            Microelectronics Center of North Carolina
 MORENET         Missouri Research and Education Network
 NADF            North American Directory Forum
 NASA            National Aeronautics and Space Administration
 .NET            Domain naming branch for network administrations



                                50




NIC             Network Information Center
NIS             Network Information Service
NNSC            NSFNet Network Service Center (NIC award prior to the
InterNIC)
NREN            National Research and Education Network (HPCC program
component)
NSF             U.S. National Science Foundation
NSFNET          National Science Foundation Network
NSI             Network Solutions, Inc.
NSP             Network Service Provider
NYSERNET        New York State Educational and Research Network
.ORG            Domain naming branch for non-profit institutions
PARADISE        European X.500 deployment project
PI              Principal Investigator
POC             Point of Contact
R&E             Research and Education community (of the U.S. in this
report)
RARE            Reseaux Associes pour la Recherche Europeenne
                (European Research Networks Association)
REU             Research Experiences for Undergraduates (NSF grant program)
RFC             Request For Comments (Internet documents, often used for
                protocol specifications)
RIPE            Reseaux IP Europeenne (European continental TCP/IP network
                operated by EUnet)
RS              Registration Services (NSI portion of InterNIC)
RWHOIS          Referral Whois protocol
SENDIT          North Dakota's K-12 Telecom Network
SIG             Special Interest Group
SIG-WEB         Special Interest Group on the WWW
SIGUCCS         ACM Special Interest Group on University & College Computing
                Services
SWIP            Shared Whois Project
.UK             United Kingdom (Domain naming branch)
.US             United States of America (Domain naming branch)
U.S.            United States of America
WAIS            Wide Area Information Servers system
WHOIS           Simple Internet white pages service (RFC 954)
WWW             World Wide Web (Internet hypertext-based multimedia
                information service)



                               51