Emerging Global Networks for Free Access to Law: WorldLII’s Strategies 2002-2005
Graham Greenleaf, Philip
Chung and Andrew Mowbray
Co-Directors, AustLII & WorldLII*
1. The emerging ‘free access to law’ network
1.1. The challenges of global legal research
Legal research increasingly has global dimensions. International trade and the WTO, the internationalisation of environmental and human rights issues, the growth of international Courts and Tribunals and the complexities of the growing numbers of regional agreements and organisations, are some obvious examples that require lawyers, governments, businesses and civil society organisations to undertake legal research with an international element more than ever before. In the processes of law reform (including the legislative implementation of treaty obligations), there is also increasing borrowing of successful models between different legal systems, particularly in those developing countries that are consciously re-building their legal systems to more closely approximate the ‘market legal systems’ of western economies. There are calls for a more international common law, and for bridges between the world’s differing legal systems1.
Commercial legal publishers have responded to these developments in a number of ways. The websites of the largest online legal publishers, particularly LexisNexis and WestLaw, are becoming increasingly global in the range of national legal materials that they offer, thus facilitating internationally-oriented legal research. Underlying this very useful development (at least for those who can afford access to these systems) is an increasing globalisation of ownership of national legal publishers. The world’s dominant legal publishing groups (Reed-Elsevier, Thomson, and Wolters-Kluwer) now own the major legal publishing houses in a large number of countries2, and are rapidly expanding to acquire more publishers in those countries and in other countries where they do not yet have a presence. Locally owned legal publishing is in decline, with legal publishing globally being dominated by these three multinational publishers.
Both the internationalisation of legal research and the decline of local control over legal information call for a response from those who value free access to legal information3, and who consider that free access to essential legal information should be part of a person’s civil rights everywhere. An international meeting of providers of free access to legal information in Montreal in 2003, declared that ‘Public legal information from all countries and international institutions is part of the common heritage of humanity.’4
By 'essential legal information' (also called ‘public legal information’) we mean primary legal materials (legislation, case-law, treaties etc) and some secondary (interpretative) materials (law reform reports, travaux préparatoires, investigative commission reports etc). The test is something like 'legal information produced by public bodies which have a duty to produce it and to make it public'.
Although there is free access to significant amounts of public legal information in the most economically developed countries, this is not the case in most developing countries. A survey of the fifty six Commonwealth countries shows that the majority of them do not have free Internet access to any significant collections of legislation or case law5.
The main goal of the free access to law movement is to spread free access to such legal information to those countries that do not have it6. A by-product of the development of local free access systems should be to prevent provision of a country’s legal information becoming the monopoly of any organisation, whether it be governments, local legal publishers, or international legal publishers.
The secondary goal should be for the local providers of free access to legal information to establish networks which facilitate searches over all of their content, so as to create a facility comparable to the global reach of the multinational legal publishers. This may seem a utopian goal, but some commentators already treat it as plausible7, and the network of free access legal information providers has grown to include almost 500 databases of legal materials in less than five years, drawn from 55 countries. This paper focuses on this secondary goal, and is largely about the World Legal Information Institute (WorldLII) project, and the context out of which it comes.
1.2. Legal Information Institutes (LIIs)
Since its mid-1990s expansion, the Internet’s provision of a relatively inexpensive but sophisticated means of access to information has provided for the first time the prospect of effective popular access to at least a country’s 'essential' law. Government agencies, Courts, NGOs, Universities, law firms, and others have created numerous web sites in most jurisdictions in the world as a result. A side-effect of this domestic ‘popular access’ is that Judges, law reformers and scholars in other countries also have such access, and those of the country in question have similar access to the laws of other countries. Provision of free global access to a country’s laws is reciprocated by free access to the laws of other countries.
Despite this optimistic sketch, the mere proliferation of legal web sites has not provided a satisfactory answer to the need to conduct systematic and comprehensive legal research, either nationally or globally. The existence of these numerous websites may not be known to users, and they are likely to have inconsistent means of both browsing and searching (if searching is even possible). After ten years of free access to law on the Internet, what we mainly find is a multiplicity of ‘silos’ of what are usually small parts of national law, with no means of linking them into a comprehensive research facility. The problem is of course exacerbated if you need to find the law from more than one country or jurisdiction.
What solutions are possible at both national and international levels? The use of search engines based on web spidering is part of the answer, as is the development of standards for legal information, and possibly the use of distributed searches8 . Another answer, perhaps more successful as yet, has been the attempts to create comprehensive law sites for all the essential law from a jurisdiction, country or region, through aggregation of data from multiple sources. There are some governmental attempt to create such comprehensive sites (noted below), but the attempt has been made more often by independent, often University-based, Legal Information Institutes (LIIs).
We use the term 'Legal Information Institute' (LII) to refer to a provider of legal information that is independent of government, and provides free access on a non-profit basis to multiple sources of essential legal information9, including both legislation and case law (or alternative sources of jurisprudence). Ideally, a LII should attempt to cover at least the most important sources of essential legal information for its jurisdictions, and should allow simultaneous searches of all its databases (so that it is not a collection of isolated databases). Systems which approximate these criteria can reasonably be called 'LIIs'10. They are therefore, in essence, aggregators of public legal information at a national or sometimes regional level.
The twelve LIIs whose collaboration is the focus of this article (with their academic bases and start dates noted) are: the Legal Information Institute (LII (Cornell) – Cornell University - 1992) for US federal law; the Australasian Legal Information Institute (AustLII – UNSW and UTS -1995) for Australia; the British & Irish Legal Information Institute (BAILII – Institute of Advanced Legal Studies, London and University of Cork - 2000) for all the jurisdictions of the UK and Ireland; the Canadian Legal Information Institute (CanLII University of Montreal - 2000); the Pacific Islands Legal Information Institute (PacLII – University of the South Pacific - 2001) for fourteen island countries of the Pacific, the Hong Kong Legal Information Institute (HKLII – HKU - 2002) for Hong Kong and potentially other parts of China; the Southern African Legal Information Institute (SAFLII – Wits University– 2003) for South Africa and potentially other countries, JuriBurkina (2004) for Burkina Faso, New Zealand Legal Information Institute (NZLII – VUW and Otego University - 2004) and Cylaw (2004) for Cyprus. A linguistic rather than regional focus is taken by Droit Francophone (2003), with initial content concentrating on West and Central Africa. The more established (ie pre-2003) LIIs are the dominant providers of free access legal information in their jurisdictions, and AustLII and PacLII are the dominant online providers per se.
As detailed in this paper, all of these LIIs, and other legal information providers, are now cooperating to provide the World Legal Information Institute (WorldLII - <www.worldlii.org>).
The logo bar used by the cooperating LIIs (example from WorldLII)
There are a small number of similar systems which provide information satisfying the definition of a LII, without necessarily using the name ‘LII’ or being involved in networking or collaboration as yet11. These include the Unidad de Documentación de Legislación y Jurisprudencia in Mexico12, the Zambian Legal Information Institute (ZamLII) (1996)13 and (on a lesser scale) the Nigeria Law system14. In many jurisdictions there are extensive but not comprehensive sites run by the government. In civil law countries most do not contain caselaw but may contain other forms of jurisprudence as well as legislation. These include systems in Argentina, Brazil, Spain, Austria, Belarus, Denmark, Estonia, Switzerland, Sweden and Romania15. Legifrance in France is becoming a comprehensive free access government provider of legislation and case law. Throughout the world there are hundreds of less comprehensive free Internet sources of caselaw16, legislation17 and other essential legal information.
Therefore, ten years after the creation of the first LIIs there are still only a minority of jurisdictions around the world that have developed comprehensive free access to essential legal information, either through creation of government sites, or by providing the data to independent free access publishers such as LIIs. As Poulin points out, most of the independent LIIs have developed in Anglophone and common law countries, and this has its dangers for the civil law world18.
1.3. The free access to law movement
Over the last few years an international ‘free access to law movement’ has started to emerge out of informal cooperation that has developed for more than a decade between its leading participants and institutions. This development has been centred around the University-based Legal Information Institutes (LIIs), but is not restricted to them, and is now expanding beyond them. So far, the characteristics of this movement have been:
It is based on independent national, regional LIIs from both developed and developing countries;
It involves the provision of assistance from the more established LIIs to jurisdictions that would like to establish new LIIs or otherwise improve local free access to law;
It is creating global legal research facilities through networks of LIIs;
It is articulating its goals through the Declaration on Free Access to Law; and
It is encouraging research and technical assistance through an annual Meeting of LIIs and the ‘Law via Internet’ Conference,
Consolidating cooperation - Annual meetings and the ‘Law via Internet’ Conference
The methods of cooperation between free access law providers are still emerging, but have developed rapidly since 2000. In 1997 and 1999 AustLII hosted the first two Law via Internet Conferences in Sydney, where informal discussions on cooperation took place between University-based web services from Australia, Canada, the USA and Norway. Some attempts were made to establish an ‘Association of Public Legal Information Institutes’ but proved to be premature.
The first sustained attempt at collaboration to build some form of international network took place at the at the LII Workshop on Emerging Global Public Legal Information Standards19 held at Cornell Law School in July 2000, and involving participants from the US, Canada, Australia and South Africa. At that meeting, the name ‘WorldLII' (for ‘World Legal Information Institute) was used to describe the challenge of developing a global free access legal research facility. Various possible models were discussed at the Cornell workshop, the most detailed of which was a distributed search system described by Cornell’s Tom Bruce20.
In 2000-2001 AustLII provided its search engine and hypertext mark-up software to assist the creation of the British & Irish Legal Information Institute (BAILII), the Canadian Legal Information Institute (CanLII), Pacific Islands Legal Information Institute (PacLII) and the the Hong Kong Legal Information Institute (HKLII). By the time of the 3rd Law via Internet Conference in Sydney in November 2001, these developments made it possible for AustLII, with the cooperation of the other LIIs, to develop and launch the first version of WorldLII, based on making all of the content of these five LIIs, plus some South African materials, simultaneously searchable. The initial (and current implementation) of WorldLII does not rely as much on distributed searches as Bruce's model, but instead uses a combination of distributed searches (for CanLII content), centrally hosted content (databases hosted on WorldLII, described later and hosting during ‘incubation’ of SAFLII and NZLII) and regular synchronisation of database copies with WorldLII (all other LIIs, plus the Legal Scholarship Network (LSN)).
The next key development took place in Montreal in October 2002, when LexUM (the operators of CanLII) hosted the 4th Law via Internet Conference at the University of Montreal, making it an annual event and for the first time held out of Australia. LexUM also initiated a day-long meeting of LIIs on the day before the Conference, both to discuss cooperation between existing LIIs and also to encourage the development of free access law services in developing countries. Interested organisations from West Africa, the Caribbean, and the francophonie attended, plus representative of all the established LIIs. The success of the LexUM initiative was proven over the next year as new LIIs took shape in the francophonie. At the Montreal meeting, the Declaration on Free Access to Law was also drafted and adopted (see next section).
The 5th Law via Internet Conference returned to AustLII in Sydney in November 2003. The Meeting of LIIs preceding the Conference followed the Montreal pattern of involving representatives of as many developing countries as possible (in this case, Cambodia, the Northern Mariana Islands, the Federated States of Micronesia, and Burkina Faso), and representatives of aid agencies (in this case, AusAid21, and the International Development Law Organisation (IDLO)). Representatives of almost all the established LIIs attended, including the newly established Droit Francophone (established by LexUM) and the Southern African Legal Informaton Institute (SAFLII) (established jointly by WITS Law School and AustLII). Another step forward was the first LII Technical Workshop, enabling groups of LII staff to concentrate on common technical problems.
In November 2004 the 6th Law via Internet / Internet pour le droit Conference was the first to be held in Europe, hosted in Paris by a number of French associations with interests in free access to law and Juriconnexion22. It followed most of the now established approach, with a LII meeting (chaired by LexUM), followed by the public Conference (with a European emphasis). JuriBurkina and NZLII formally joined the free access to law movement at the Confernce, and Cylaw did so shortly afterwards. Minor amendments were made to the Decalaration, and a protocol for the sharing of case citation tables between LIIs was agreed. New hosts for future Conferences and LII meetings will, we hope, assist the global spread of free access to law. PacLII will host the 2005 Conference (with AustLII assistance) at the University of the South Pacific (USP) in Vanuatu.
The free access to law movement has not established any formal organisation, seeming to prefer ‘rough consensus and running code’23. Between LII meetings it communicates via email, and the Chair of the previous LII Meeting hands over responsibility for broadcast email communications to the Chair of the next year’s Conference around mid-year.
In the five years since the Cornell meeting, the kernel of a global free access to law movement has therefore emerged, and is establishing principles, forums and methods of cooperation, and techniques for significantly expanding the global reach and effectiveness of free access to law. The challenge now is to engage with people in many more countries in both the developed and developing worlds, in keeping with the goals of its Declaration on Free Access to Law.
1.4. Public policy and free access to law
1.4.1 The Declaration on Free Access to Law
In October 2002 the meeting of LIIs in Montreal at the 4th Law via Internet Conference, made a declaration24 as a joint statement of their philosophy of access to law. There were some further modifications of the Declaration at the Sydney and Paris LII meetings25. The ‘consolidated’ Declaration is as follows:
Legal information institutes of the world, meeting in Montreal, declare that:
• Public legal information from all countries and international institutions is part of the common heritage of humanity. Maximising access to this information promotes justice and the rule of law;
• Public legal information is digital common property and should be accessible to all on a non-profit basis and free of charge;
• Independent non-profit organisations have the right to publish public legal information and the government bodies that create or control that information should provide access to it so that it can be published.
Public legal information means legal information produced by public bodies that have a duty to produce law and make it public. It includes primary sources of law, such as legislation, case law and treaties, as well as various secondary (interpretative) public sources, such as reports on preparatory work and law reform, and resulting from boards of inquiry. It also includes legal documents created as a result of public funding.
A legal information institute:
• Publishes via the internet public legal information originating from more than one public body;
• Provides free, full and anonymous public access to that information;
• Does not impede others from publishing public legal information; and
• Supports the objectives set out in this Declaration.
All legal information institutes are encouraged to participate in regional or global free access to law networks.
Therefore, the legal information institutes agree:
• To promote and support free access to public legal information throughout the world, principally via the Internet;
• To cooperate in order to achieve these goals and, in particular, to assist organisations in developing countries to achieve these goals, recognising the reciprocal advantages that all obtain from access to each other's law;
• To help each other and to support, within their means, other organisations that share these goals with respect to:
• Promotion, to governments and other organisations, of public policy conducive to the accessibility of public legal information;
• Technical assistance, advice and training;
• Development of open technical standards;
• Academic exchange of research results.
• To meet at least annually, and to invite other organisations who are legal information institutes to subscribe to this declaration and join those meetings, according to procedures to be established by the parties to this Declaration.
• To provide to the end users of public legal information clear information concerning any conditions of re-use of that information, where this is feasible.
The essential elements of the Declaration can be summarised as:
• The demand that governments do not exercise any monopoly control over ‘public legal information’ and that they make it available to independent parties who wish to republish it. We refer to this as ‘full free access’ and it is discussed further below.
• The agreement between LIIs to cooperate with each other.
• The agreement between LIIs to adhere to the principles of the Declaration.
The emphasis is on the support for free access. The Declaration does not require participation in any networks, but does encourage such participation, and all existing members do in fact participate in the WorldLII and/or Droit Francophone networks.
1.4.2. Public policies to maximise access to law: ‘Full free access’
The Declaration states:
Independent non-profit organisations have the right to publish public legal information and the government bodies that create or control that information should provide access to it so that it can be published.
To amplify that policy, we might ask whether it is sufficient if governments provide free access through government-run websites? We have argued (on behalf of AustLII) since 1995 that this is not enough26, and that official bodies should accept that they have seven obligations in the provision of essential legal information if they are to give optimal support to the rule of law and other values:
1 Provision in a completed form, including additional information best provided at source, such as the consolidation of legislation, and the addition of catchwords (index terms) or even summaries to cases.
2 Provision in an authoritative form, such as use of court-designated citations for cases and (eventually) use of digital signatures to authenticate the versions distributed.
3 Provision in the form best facilitating dissemination, which should always now mean in electronic form, should in most cases be possible by email or more sophisticated forms of data delivery, and should be possible in a form facilitating conversion.
4 Provision on a marginal-cost-recovery basis to anyone, so that governments do not attempt to profit from the sale of public legal information, thereby creating artificial barriers to access to law.
5 Provision with no re-use restrictions or licence fees, subject only to such minimal restrictions as are necessary to preserve the integrity of published data27.
6 Preservation of a copy in the care of the public authority, so that an archive of the data is preserved to enable greater competition whenever a new entrant wishes to publish the data, whether or not the public authority publishes the data itself.
7 Non-discriminatory recognition of citations, so that Court-designated citations are not removed from ‘reported’ cases, so that the privileged status of citations of 'official' reports is reduced or eliminated28.
A corollary of these propositions is that it is not sufficient for official bodies only to publish essential legal information for free access on their own web sites. Provision to other publishers (including both LIIs and commercial publishers) is also necessary for sound public policy, and is more important than official self-publication. Such dissemination is necessary to ensure that free access is not second-rate access. ‘Full free access’ is therefore not based so much on the public’s free access to any particular legal information system but rather on public bodies accepting an obligation to provide free access to the computerised sources of legal data to those who wish to republish it. Governments may or may not publish the information themselves, but competition will help ensure that one or more versions are available for free access. In the words of the Free Software movement, it is based on ‘free as in free speech’ , not on the provision of any particular brand of free beer.
The spirit of these AustLII public policy principles is now reflected in the Declaration on Free Access to Law, but the detailed elaboration of what ‘full and free access’ implies is also worth undertaking if the full implications of the free access to law movement are to be understood. These wide-ranging propositions cannot be explored fully in this paper29.
2. Conceptualising global legal research
Before proceeding to a discussion of the structure of the free access network that is emerging, or WorldLII’s role in it, it is worth asking what is meant by ‘global legal research’ in the sense of ‘what are the different bodies of data that we might be interested in accessing?’. There is not much point doing this in an a-contextual or a-historical fashion. There is now a decade’s development of Internet legal research which has established a landscape in which we must now work. First, a relatively small number of Legal Information Institutes have contributed a disproportionate quantity of the good quality legal information available on the Internet, and are establishing networks between them. Second, general Internet search engines and Google’s current predominance among them, and its enormous reach, must be considered. Powerful general Internet search engines are the context within which any other services must be designed.
2.1. The visible and the hidden legal webs
From this perspective, at least the following seven categories of legal information relevant to the development of a global system of legal research include must be considered:
Legal Information Institute content – This is generally high quality legal content that LIIs have been able to aggregate and republish in their own databases. Access to such data is commonly shared through search interfaces provided by other LIIs, even if robots are otherwise generally excluded from the LII site so as to prevent searching from general Internet search engines30.
Content spiderable by LIIs – This is the content of legal websites that allow web spiders to index them, and are significant enough to be within reach of law-specific web spiders run by a LII (such as WorldLII or Droit Francophone). Because a LII has to use a small number of human indexers to decide which sites to send its web spider to, such indexes will inevitably be limited in size (sites may be too obscure or simply missed, or the legal content may be a minor part of a site with mainly non-legal content).
Content spiderable by Google etc – Legal content on websites that allow web spiders to index them will quite possibly be searchable through general search engines such as Google, but may have not been indexed by any LII web spider with their lesser reach.
Sites as yet missed by spiders - Legal content on websites that have simply not yet been indexed by any web spider, whether a law-specific one from a LII, or a general one such as Google, may diminish as Internet search engines become more powerful.
Sites excluding spiders – Some legal websites exclude robots from indexing part or all of them, either for performance reasons or to retain control over the methods of access to their data. They do so not by technical means but through use of the Robot Exclusion Protocol31, with which search engines generally comply voluntarily.
Sites resisting spiders Other legal websites are resistant to web spiders for technical reasons, because of factors such as dynamically generate pages, or sites that only provide database searching but no tables of contents for browsing, or because of passwords preventing entry of spiders, or because of unusual file formats. The proportion of sites that resist spidering for these types of factors rises and falls, as on the one hand search engines create more effective spiders, and on the other hand web technologies change.
Subscription-only web sites – Sites on which information is only available for a fee (including both commercial publishers’ sites and those of some government providers) or to certain categories of members32, will normally exclude web spiders33.
Categories 4-6, or some similar categorisation, are sometimes referred to as the ‘hidden web’ in the sense that they are not visible to anyone relying on Internet search engines to find information34. This is something of an exaggeration, because it may still be possible to find the information by relying on a human-developed catalog such as Yahoo!’s catalog or a LII law-specific catalog (such as the WorldLII or Droit Francophone catalogs). How much legal information is in each of categories 4-7 is not known. To the extent that categories 4-6 are significant, Internet catalogs for law still have a valuable role to play. More research is needed on this.
Another major factor is the language(s) in which the information on the site is available. As Poulin observes, people do not generally want to do legal research in any language, but only in the language or languages they understand. Identification of which sites are in which languages, and handling web spidering and searching problems presented by different languages, adds an additional layer of complexity35.
2.2. Implications for global legal research tools
Some of the implications of these seven categories for anyone wanting to design a global legal research system are as follows:
A LII which aims to be comprehensive (whether globally, nationally, linguistically, or in relation to a particular subject) will never be able to aggregate in its own databases all the content that its users need for their legal research. It therefore has to consider what databases it can provide itself, what it can do to provide access to external data, and how best to integrate the two.
If content is available in a LII’s own database (or from a collaborating LII), it will be able to exercise a high degree of control over its consistency, presentation, how it can be combined with other data, and generally in the degree of value-adding that is possible.
Data obtained by web spidering may be of quality equal to LII data (but there are fewer guarantees that it will be) but is less tractable. Its consistency and quality can be improved by encouraging adoption of standards, but over the last decade most attempts to develop standards for legal information which will be applied by organisations which do not have otherwise strong reasons to work with each other, have made little progress36.
Sites excluding spiders, or resisting spiders, may contain some of the highest value legal content, as they include many large databases. LIIs may need to be willing to make special effort to obtain and update their data. These could include:
• negotiating special arrangements with these sites to waive robot exclusions, or provide concordances, so they can be searched with other external sites; or
• manual extraction if the data is important enough37; or
• search brokering, where the LII sends searches to the other site for its users, then merges the results with results it obtains from its databases or web spidering; LIIs have not yet explored this approach very much, and the results have been mixed in terms of response times38;
When a law-specific web spider is developed, the human effort that is required to ‘target’ the web spider accurately will also produce a catalog of web sites which can easily also include web sites resistant to spidering, making it more comprehensive. Both WordLII (since 199739) and Droit Francophone (since 200340) have adopted this approach. The classificatory work involved in targeting / cataloguing means that it is possible to impose some control over how the results can be used to create searches of limited scope.
The work required to target law-specific web spiders means they cannot ever approach the vast coverage of Internet search engines such as Google. It is therefore necessary to also consider how to utilise these general Internet search engines for the benefit of the legal research needs of LII users.
Similarly, much valuable legal information is only available from commercial legal publishers. It would be highly desirable if LII users could move seamlessly from their LII research to be able to access those commercial resources to which they have access (if any).
These factors have led us to design WorldLII as a combination of five facilities, as will be discussed in detail in the latter parts of this paper:
The databases located on WorldLII or its collaborating LIIs are the highest quality component, capable of significant value-adding. A variety of methods are used to make these databases searchable from WorldLII.
A human-generated Catalog of external legal websites is maintained, with global coverage. This facility assumes that such catalogs are still worth maintaining in law, an assumption which needs to be questioned at regular intervals as the performance of general Internet search engines change. At this point such a Catalog is justified by our qualitative assessment that a sufficiently high proportion of significant legal sites resist or exclude general Internet search engines, so catalogs are needed as a supplement.
The Catalog is used to target a web spider to index those law sites in the Catalog that are capable of being spidered, creating a Websearch facility. This is a lower quality search facility, less amenable to value-adding. Its maintenance is more questionable, in light of the growing coverage of general Internet search engines, but it is maintained to enable further research on alternative methods of relevance ranking to those used by Google.
A facility to automatically translate user searches over WorldLII into searches over Google, and to limit the scope of the searches to legal materials, augments the limited scope of our own web spider.
On AustLII, a Publishers Search facility has been added, allowing users to repeat their search automatically over the full text collections of various legal publishers41. This element is not yet under development for WorldLII.
Droit Francophone, the second hub to be developed in the emerging global legal research network, has also been designed as a combination of the first three of these facilities.
2.3. A decentralised global network
The global spread of free Internet access to law is separate from and not dependent upon the participation of the free access websites in any global network. The argument of this paper is that free access legal websites become more valuable to their users by becoming interconnected with others to share their data, or (if that is not technically practical) by allowing their data to be republished on a LII which is part of the global network. However, free access, not networking, is of primary importance.
The global free access network that is emerging can be generally described as follows.
It is based on independently operating local Legal Information Institutes at a national or sometimes regional level, which are essentially aggregators of public legal information from that country or region.
In many cases these local LIIs have been set up with the technical assistance of one of the longer-established LIIs, with the provision of software it has developed, and sometimes with the initial use of its infrastructure. As a result there is at this stage considerable consistency in appearance of pages within the network, and common software and infrastructure which facilitates networking.
In some cases, the content found on a LII will be the only version of that content available on the Internet42, or the only version available for free access. However, in many cases the content available on a LII will duplicate (republish) some of the content found on another web site, often sites that are substantial legal web sites in themselves. For example, PacLII is republishing the content of already successful local free access facilities in the Federated States of Micronesia; in Australia all governments publish free access legislation online, but AustLII republishes all jurisdictions’ legislation in a consistent format.of
As a result, the network has very distributed data collection and data maintenance responsibilities43.
There are a number of ‘hubs’ in the network which aggregate access to the content of particular groups of participating LIIs in the network. The hubs operate on a linguistic or geographic (regional / national) basis. There are a small number of hubs: Droit Francophone (French language basis), WorldLII (primarily an English language basis)44, and PacLII (a Polynesian and Melanesian cultural hub45). Other linguistic or geographic hubs should emerge over time. HKLII has the potential to develop into a broader Chinese language hub, and SAFLII could become a hub for Anglophone countries in the region.
Hubs pay attention to the interface needs of particular user communities (eg French-speaking, English-speaking), and develop different facilities for presenting the same data. Each hub is therefore a different view or interface to the whole of the data in the network (including by selecting from the data to only present part).
Wherever possible, content is located on one of the national/regional LIIs rather than on one of the hubs, and wherever possible on a language-appropriate hub, so as to maximize decentralisation. For example, where WorldLII has initiated projects to aggregate decisions of international Courts and Tribunals, or Domain Name Decisions, or Privacy and FOI databases, the appropriate LIIs are asked whether they wish to host any of the databases appropriate to their LII. If they do not, or do not at present, then WorldLII hosts the database.
The hubs are interconnected in a variety of ways, creating a global free access network. More accurately, a number of global networks access the same pool of shared data.
LIIs and hubs not only share access to their database content, but are also developing means of sharing their facilities to assist research beyond the LIIs such as catalogs and websearch facilities. For example, the WorldLII Catalog is shared by BAILII, PacLII, SAFLII, HKLII, NZLII, AustLII and WorldLII, and is accessible from each of those sites. The Droit Francophone catalog could also be shared by Juri Burkina and other francophone sites, but is not as yet.
There is no single central hub, or apex, of the whole network, only different views of the shared content or selections from it46.
The network will become much more decentralised over time as more LIIs and hubs emerge (5 LIIs in 2001, 9 in 2003, 12 in 2005).
The development of standards for presentation and interchange of information will be of much greater importance in the development of the network in future, as the significance of de facto standards caused by common software and infrastructure decreases. 47
From the perspective of where data or concordances are located and accessed, global load balancing and redundancy will reduce the distinction between centralisation and decentralisation.
Irrespective of the extent to which they grow, LIIs and the network of LIIs are only likely to cover a modest part of the free law universe. As a result, for global legal research facilities to develop, the role of tools reaching beyond the LIIs, such as catalogs, websearch, and methods of extracting law from Internet-wide search engines will continue to be of importance.
In summary, we would describe this structure as a modestly decentralised network.
A graphical representation of the emerging free access to law network as it existed in November 2003 is below. However, it changes rapidly, and this diagram does not include SAFLII, NZLII, or Cylaw, all of which are networked via WorldLII. The balance of this paper details the operation and role of one of the hubs in this network, the World Legal Information Institute (WorldLII).
Free access to law network, c. November 2003
3. WorldLII’s role - a locus of five strategies
The World Legal Information Institute (WorldLII) <http://www.worldlii.org > is one of the hubs of this network. It serves five distinct purposes and is not so much a single service but a locus of five strategies to advance global free access to law:
An International LII
An Incubator of LIIs
An Integrator of LIIs
An Interface in English to LII content … and a bit more
A Platform for global research beyond the LIIs
Each of these strategies is discussed below, following a brief outline of WorldLII’s operational aspects.
WorldLII front screen (extract omitting logo bar), May 2005
3.1. Operations – infrastructure, software and funding
WorldLII is operated by AustLII, which is a joint facility of the Faculties of Law at the University of Technology, Sydney (UTS) and the University of New South Wales (UNSW). AustLII and WorldLII share the same premises, hardware and staff (currently 10, of whom less than half are likely to be working on WorldLII matters at any given time).
On a day-to-day basis, AustLII runs the WorldLII facility. However, as a hub in the LII network, WorldLII only exists because of the collaboration and cooperation of the other LIIs in the network, and there is a considerable amount of consultation with other LIIs. The prototype of WorldLII was launched at the 2001 ‘Law via Internet’ Conference, and was ‘endorsed by all the existing LII teams’48 at the LII meeting in Montreal in 2002.
3.1.1. Funding WorldLII
WorldLII was started by AustLII as an unfunded project (except for the Catalog, then called ‘World Law’ and funded by the Asian Development Bank under ‘Project DIAL’49), as an outcome of AustLII’s also largely unfunded assistance in establishing other LIIs. However, it has subsequently obtained funding as described below. This is another example, useful for those establishing LIIs in developing countries, of Poulin’s point that ‘the open access to law approach was always developed with minimal resources’50.
The overall funding for AustLII and WorldLII is now about A$1.4 million per year (2005 budget). About 40% of that funding comes from the Australian Research Council’s (ARC’s) research infrastructure (LIEF) fund, for which AustLII has to bid competitively on an annual basis. The funding for AustLII and WorldLII is only on an annual basis but has increased in each year for the last decade, by an average of A$100,000 per year. The combined staff of ten has grown by an average of one position a year for the last ten years. Part of this ARC funding, and funds from AustLII’s two host Universities, can be regarded as ‘WorldLII funding’, perhaps to A$400,000 in total51. The Asian Development Bank has in past years been the major funding source for the development of the WorldLII Catalog and Websearch facilities.
3.1.2. Technical infrastructure of AustLII/WorldLII
The technical infrastructure supporting AustlLII and WorldLII is based on a V880 Sun Fire sever with 64 gigabytes of memory and 8 UltraSPARC III CPUs. The raw capacity of the main storage unit is one terabyte. The main server is supplemented by a Sun Enterprise 4500 dual processor server plus 210 gigabytes of RAID array storage. A set of 8 blade servers are used for load balancing and related purposes with the capacity to add another blade server. Many other SPARCs (of varying vintages from SPARC 1’s up) are used for specialised purposes. The main servers are at UTS, and there is a synchronised backup server located offsite at UNSW. About 40% of each annual budget is spent on equipment upgrades (ie A$400,000 in 2004), necessary to keep pace with increasing usage and the expansion of the AustLII and WorldLII data sets.
AustLII and the WorldLII hosted databases and WorldLII Catalog receive at least 560,000 hits per working day (April 2005), peaking at around 800,000 hits per day, with services and content hosted on WorldLII (not other LIIs) accounting for around 10% of that. However, most usage of WorldLII involves searches over the collaborating LIIs, and we have not measured the additional hits that the contributing LIIs receive as a result of access via WorldLII searches or browsing. AustLII and WorldLII usage is probably considerably higher than these internally-generated figures suggest, because of the prevalence of proxy and cache servers, which reduce drastically the number of hits measurable by AustLII on popular cases and the like. The ‘hidden proxy/cache’ multiplier is unknown.
The AustLII/WorldLII servers also host synchronised copies of the BAILII, PacLII, HKLII and SAFLII databases and concordance, for purposes both of backup security, and to enable global load balancing to be established. AustLII/WorldLII therefore has very significant data storage needs for this purpose. The copy of PacLII in Sydney also acts as the principal server for PacLII users outside the Pacific, due to slow Internet access speeds to Vanuatu. At present, until, migration is complete, the Sydney copies of SAFLII and NZLII are the only copies.
3.1.3. AustLII’s shared software suite – a strategic foundation of WorldLII
AustLII is based on the development of its own software52, principally the following:
• The SINO53 search engine provides very fast retrieval with Boolean and proximity operators, and relevance ranking of search results. The ranking algorithm is ours54, and is currently being enhanced to take into account hypertext links between documents in our databases as an indicator of significance of documents55. An Autosearch function anticipates the type of search a user wishes to carry out, from their search terms, and attempts to automatically apply the correct search type (Boolean, ‘any of these words’, title search etc)56.
• The suite of hypertext mark-up tools including FindActs and FindCases, and tools for splitting complex documents such as Acts into their component parts57. These tools are used to automatically insert approximately 30 million hypertext links within the AustLII and ‘hosted on WorldLII’ databases. The legislation tools have been enhanced to allow the construction of point-in-time legislation58.
• A set of web cataloguing and web spidering tools allow the construction of the WorldLII Catalog and Websearch facilities59. These tools do not need to be provided to other LIIs because only one WorldLII Catalog is maintained, with other LIIs contributing to its content development and sharing access to it.
This suite of software is used to run WorldLII and AustLII, and the software, and its updates, are provided at no cost, and with source code, to run BAILII, HKLII, PacLII and SAFLII. BAILII and HKLII have already made their own modifications and additions to the software suite. LexUM used its modified version of the SINO search engine on CanLII and Droit Francophone until 2004, but has now developed its own suite of software based on open source tools60. All of the AustLII tools have been developed in a Unix environment, but LexUM successfully ported SINO to a Windows environment.
Since 1999 the AustLII/WorldLII software suite has been provided free of charge, with such assistance from AustLII in its technical implementation as we are able to provide, to organisations from any country that meet the following criteria61:
It has a commitment to providing free access to legal information, to all persons without conditions62. We accept that the same service may need to have some non-profit paid access to help fund its provision of free access services, provided this does not undermine the provision of free access to any public legal information.
It endorses the Declaration on Free Access to Law, and does not intend to do anything to undermine the goals of that Declaration63.
It has the capacity to obtain significant quantities of legal data and to develop a system technically, and has reasonable prospects of financial sustainability. These factors help ensure that AustLII’s provision of technical assistance is not wasted, and is within the capacity of our own resources at the time.
A willingness to be part of the global free access network , either by becoming a LII participating in the data sharing of that network, or by allowing republication of its data via an existing LII. This might not be necessary in all cases. All current LIIs participate in WorldLII.
Suitable local partner organisations to date have turned out to be Universities or University-based. However, this is not necessarily so, and government or civil society bodies or even some otherwise commercial organisations could fit these criteria. Our guiding principle is to avoid undermining the development of free access in a jurisdiction by providing software and assistance to those who are not supporting full free access. Where an organisation is willing to allow republication of its own data via an existing LII, but would like to use the AustLII/WorldLII software for its own system which does not meet some of these criteria, we would often also be willing to supply the software on that basis.
We are therefore using the AustLII/WorldLII software as a strategic asset to advance the cause of global free access to law, an approach that can be described as ‘shared software’.. It is neither ‘free software’ nor ‘open source’ software in the terms of those movements, even though the source code is shared with those to whom it is provided, with liberty for them to develop it further. In effect, AustLII has a ‘free for free access’ licence but it has not been formalised.
If we had made the AustLII/WorldLII software open source in the early days of web take-up a decade ago, we probably would have better software by now, but the most likely organisations who would then have had the capacity to use it to provide access to legal information would have been governments and commercial legal publishers of various sorts, with no commitment to providing free access to legal information, much less the full free access that allows republication by others. Our strategic assessment was that we would undermine the goals of spreading global full free access by making the software open source. Instead, we have been committed for at least the last five years to provide it to anyone who genuinely wanted to advance free access.
At some point, the strategic benefits of our approach may be outweighed by the considerable advantages of open source software development64, but our assessment is that, at least in relation to many countries, that time has not yet come. LexUM is now taking the open source route with the suite of software it is developing65, and it will be very valuable to have both approaches operating within the free access to law movement.
We consider that AustLII and WorldlII (and other LIIs) are part of the ‘commons movement’, but the commons that we are primarily interested in is that of legal content, not of software. Our approach is that commons, to be valuable and effective, require organisation of the type that LIIs provide effectively. That, too, is a longer argument and will not be pursued here66.
3.2. Strategy 1 - An International LII
As well as being a single point for searching the content of the other collaborating LIIs, WorldLII is a Legal Information Institute in its own right, with a growing collection of databases. The main focus of WorldLII’s own databases is on international law, with the main content at present being decisions of international Courts and Tribunals and some internationally-oriented journals, plus some databases which deal with subject-specific legal developments which do not fit on any other LII (eg domain name .ccTLD databases).
3.2.1. Decisions of international Courts and Tribunals
The International Courts and Tribunals Project menu67, shown below, shows databases of final decisions from twenty three international Courts and Tribunals68, of which two are located on Droit Francophone, two on BAILII, and the rest on WorldLII. These databases and substantially complete, containing the full set of historical decisions of each Court, but only final not interlocutory or interim decision. This is by far the largest searchable collection of such decisions on the Internet.
Where Courts provide all their decisions in French, copies are now being captured and prepared, and will be located on Droit Francophone with cross-links between each decision located on the two systems. These matters are worked out in a pragmatic way by the cooperating LIIs, depending on who has the resources to do the work at the time.
International Courts and Tribunals menu (extract)
From the bottom of this page the link to the International Courts and Tribunals page of the WorldLII Catalog and Websearch provides links to the web pages of all other international tribunals , and the websearch facility is able to make some of their decisions searchable.
3.2.2. Other international collections
At present there are only a small number of internationally-oriented law journals on WorldLII69, plus the abstracts of the articles and working papers contained in the Legal Scholarship Network (LSN), which has over 17,000 items. AustLII has twenty Australian and New Zealand law journals in full text, but other LIIs have few as yet. This is an area of possible expansion for LIIs, as it is an area which can be regarded as public legal information, and the extent to which commercial publishers attempt to assert exclusive rights over the content of academic law journals is very contentious.
Another area in which WorldLII may develop its own databases is international treaties. LIIs already contains one major collection, the Australian Treaties Library (back to 1900 in full) on AustLII, a substantial collection from PacLII and an index from HKLII. The fact that the UN Treaties Series (UNTS) is only available to subscribers is a cause of concern to anyone interested in free access to law, and a reason for WorldLII to focus on collaborative LII development of an alternative resource.
3.3. Strategy 2 - An Incubator of LIIs
The second major source of databases ‘hosted on WorldLII’ are those which are not intended to reside permanently on WorldLII, but rather are being developed there as the first step toward ‘incubating’ a new Legal Information Institute, often in a developing country but sometimes not. Thus AustLII assisted the development of BAILII, PacLII and HKLII, and in each case ran the initial implementations of those systems on its own servers in Sydney for one to two years while our local partners (in the UK, Vanuatu and Hong Kong) set up the necessary administrative structures, financial resources, and technical infrastructure to set up their own local servers and take over day-to-day maintenance and development of the system. The AustLII server still sometimes serves a backup function. This process of ‘technology transfer’ has worked very well: each of the systems is very successful and is now completely independent of AustLII, though still using its software.
WorldLII now gives AustLII a more flexible and incremental way to provide such assistance to developing countries, because it provides a logically organised database structure in which we can host databases from any country. We are willing to include on WorldLII databases from any country (subject to resource limitations) where this will assist the development of free access facilities across the world. However, our preference is only to do this on a temporary basis until a suitable local host LII can take over the databases. We also have priorities to obtain databases from Asian countries and from Commonwealth countries, in keeping with Australian research priorities as expressed in our funding applications.
LexUM, operators of CanLII, have developed their own methods of assisting developing countries to create free access to law facilities which are proving to be equally successful70. In 2004 Juri Burkina was launched, a legal information institute for Burkina Faso built with LexUM’s assistance. LexUM has also developing a generic platform comparable to WorldLII with Droit Francophone, by which it is developing legal databases for all the countries of the francophonie, with an initial emphasis on those in Central and West Africa71. Although it is done in a technically different manner, which is very good in that it allows for more transparent and immediate uploading of files from local contributors than does WorldLII, the outcomes may be similar in some respects. Juri Burkina is already becoming an independent LII separate from (but integrated with) the Droit Francophone structure. It will not be surprising if other LIIs ‘incubate’ from the Droit Francophone structure in the course of time.
3.3.1. Examples of ‘incubation’
SAFLII A good example of how this process works is South Africa. The University of the Witwatersrand Faculty of Law (Wits) has for many years maintained the best free access collection of legal databases in South Africa, comprising major national courts and law reform reports. Wits did not, however, hold legislation, provincial cases, or a search engine which could search all of its collections simultaneously. From 2001, Wits agreed to provide all of its databases to WorldLII, so they could be hosted on the South Africa directory of WorldLII while we all gained experience in using the AustLII software with this data, established procedures in sending and marking up data, and gathered sufficient data to make a separate LII feasible. Between 2001-2003 the South Aftrican databases were simply ‘hosted on WorldLII’. At the end of the 2003, Wits decided it was ready to launch the Southern African Legal Information Institute (SAFLII), the front page of which is below. The content does not yet rival the coverage of other LIIs, but it is a respectable basis for a separate system: the leading national courts, national legislation, the Law Commission’s reports, and the Southern African components of the WorldLII Catalog. WordLII search results now list any South African content as coming from SAFLII, not WorldLII. SAFLII is now run from AustLII’s servers in Sydney, but we expect that within a year or so the infrastructure will be in place for the technical migration to be complete.
SAFLII front page (extract)
NZLII An illustration of the inherently decentralising nature of this network is the New Zealand Legal Information Institute (NZLII), a separate LII since October 2004. For some years a few New Zealand databases have been included in AustLII (thereby giving some limited truth to the ‘Australasian’ of the name), but we agreed with our New Zealand colleagues that progress in developing NZ content was likely to be faster if managed locally with a separate web identity, though the NZ content (now badged as NZLII content) also continues to be available via AustLII. AustLII will continue to run the NZLII server and provide technical support until our NZ partners, Otago Law School and Victoria University, Wellington, Law School are in a position to do so. This illustrates that this network is decentralising, not centralising.
Cambodian databases A variety of government and civil society organisations in Cambodia want to support the rule of law in Cambodia by providing free access to law. Given the tragic history of Cambodia in the last forty years, this is an immense challenge, but it is being undertaken with determination by many people and organisations72. Our involvement with Cambodia started with an Asian Development Bank project (DIAL) through which we trained government lawyers in Phnom Penh to use the Internet for legal research, with WorldLII as the basic platform for that training73. Subsequently, in order to help demonstrate what can be done on the web with Cambodian legal content, WorldLII hosts over 300 legislative items in Khmer, English & French (the largest collection available on the Internet), plus the decisions of the Cambodian Arbitration Council, the first cases ever published in Cambodia (in any form, print or electronic). The Cambodian content of the WorldLII Catalog is also extensively developed. All of these resources can be searched from the Camdodia home page in WorldLII74. We continue to collaborate with various Cambodian colleagues from government and civil society, and funding sources, seeking some sustainable form for the coordinated development of free access to Cambodian law. Cambodia will develop its own unique solution, and we hope it will become a leading example to other developing countries.
Indonesian databases AustLII is working with the Indonesian Ministry of Justice and Human Rights (funded by AusAid and the Australian Attorney-General’s Department) to assist the Ministry to develop online resources, and has trained three Ministry officials for this purpose. There are a number of outputs so far, including an Indonesian Project page75 on WorldLII which contains the full text of Indonesian national legislation and regulations from 1949-2002, in Bahasa, plus the Indonesian pages of the WorldLII Catalog in both English and Bahasa. In addition, the trainees from the Ministry are working on developing a comprehensive site for Indonesian legal information, operated independently of the Ministry, which will also incorporate access to the databases on WorldLII.
Timor-Leste databases AustLII has developed a Timor-Leste Project page76 on WorldLII which as yet only includes all of the pre-Independence UNTAET legislation (Regulations, Directives and Executive Orders) in English, the Timor-Leste part of the WorldLII Catalog, treaties between Australian and Timor-Leste, and over 80 journal articles touching on its law. We have now obtained the permission of the Timor-Leste government to include on WorldLII the post-independence legislation in Portuguese and Tetum. We are seeking funding sources and local partners in the hope that this project will develop into a separate Timor-Leste LII.
3.3.2. A training resource for LII development
Poulin states the case for LIIs to provide online resources to assist others to develop free access legal resources, particularly in developing countries77:
In order to help countries prepare and publish legal information, it would also be useful to prepare training material and courses as well as a specialized Website on how to develop open access to law Websites. An open access to law resource site could offer methodological resources, training programs, software and an online toolbox in order to cover the publishing procedures required to carry out the tasks. At this point in time, from a practical point of view, such a resource can be designed only with the support of more experienced open access to law institutes.
A modest step toward this goal is the ‘Computerisation of Law’ database on WorldLII of papers from the ‘Law via Internet’ Conferences, and other technical papers which may be of assistance78. Technical details of AustLII’s own tools will also be provided. LexUm is already taking steps along these lines79.
It is also time for the existing LIIs to consider establishing ‘Access to law training programmes’ to run intensive training, perhaps annually, for participants from developing countries (probably funded by international agencies) in all the skills needed to establish a free access to law facility. At present, these would be most feasible in English and French. In the field of international advocacy, the Diplomacy Training Programme (based at UNSW) has trained NGO representatives for many years, and could be a valuable model80. AustLII has completed one training course for Indonesian government officials, and has helped train staff at PacLII and other LIIs.
3.4. Strategy 3 - An Integrator of LII content
WorldLII’s core purpose is to be a central point from which the combined databases of all the collaborating LIIs may be browsed and searched. However, the way in which it integrates LII content is complex and evolving. The most obvious strength of WorldLII, and the most obvious form of LII integration that it provides, is that it allows nearly 500 databases (478 at the time of writing – updated on its front page) from 55 countries and International Courts and Tribunals to be searched simultaneously. This is the default scope of searchese. The countries included, and the collaborating LIIs that provide them, as yet are: AustLII - Australia ; BAILII - Britain and Ireland; CanLII – Canada; Droit Francophone - 18 countries, mainly in Central and West Africa, plus a few in Asia and Europe; HKLII - Hong Kong; LII (Cornell) - US Federal law; PacLII - 16 Pacific Island countries; SAFLII - South Africa; NZLII - New Zealand; CyLaw - Cyprus. The balance are made up of databases on WorldLII from ‘incubating’ countries (including Indonesia, Brunei, and Timor-Leste ), some Asian legislation collections, and numerous databases of .ccTLD (country code) domain name dispute decisions. For some countries there are over a hundred databases (Australia and Canada), for some there are as yet only one. The number of databases has grown at about 20% per annum since WorldLII was formed in 2002.
3.4.1. Integration by searches over database selections
Despite this breadth, the most valuable search feature of WorldLII will often be that it allows narrower searches over particular types of materials across a wide range of jurisdictions. From the Advanced Search page on WorldLII users are offered a range of pre-determined selections of databases in addition to the ‘All databases’ default. In addition, users may choose their own combinations of any of the databases accessible from WorldLII by following the instructions on the Advanced Search page81.
The pre-determined database groupings include groupings by type of document to be searched (eg ‘All legislation’, ‘All case law’, ‘International Courts and Tribunals’, ‘All journals’, ‘All law reform’), various geographic groupings (eg ‘Asia: All databases’ and so on for Africa, North America etc) and some subject-oriented groupings (eg ‘Privacy and FOI databases’, ‘Domain Name decisions’). Some of the more important or well-developed groupings of searchable databases on the ‘Advanced Search’ page are also the basis of the ‘Project’ pages accessible from the left column of WorldLII’s front page. The advantage of the ‘Project’ pages is that the user can see a full list of the databases included in the selection, and also choose to modify it. We think that these searches of more limited scope than the whole collection, both from the Advanced Search page and various Project pages, are the best examples of how the cooperation between LIIs can be used to develop very powerful and very targeted research resources. We will now give a few illustrations.
Legislation example A search over ‘All Legislation Databases’ from the Advanced Search page searches legislation databases from over 20 countries and 40 jurisdictions. For example, a search for ‘copyright near circumvent*’ finds copyright legislation implementing anti-circumvention provisions (influenced by the WIPO Copyright Treaty) from seven countries. As the search result below illustrates, for most of the legislation (UK, Ireland, Australia, USA, Hong Kong) the search result links to the precise section or sections of the Act or Code prohibiting circumvention of technical protection measures, and from the others (Fiji and Samoa) it goes to the Act (because PacLII does not break its legislation into sections). The seven results displayed below show the most relevant result from six of the seven jurisdictions, with the rest of the results showing other relevant results from the same jurisdictions (plus Samoa). We consider this compares well with legislation searching on comparable services.
Example search result over ‘All Legislation Databases’
Law reform example To search over the reports of all law reform bodies available through WorldLII, a user can choose ‘WorldLII: All Law Reform Databases’ from the Full Search Form above, or alternatively go to the ‘Law Reform Project’ page shown below, which has the advantage that the available law reform bodies are listed. As can be seen, the full text of the reports of the law reform Commissions of Australia, Hong Kong, Ireland, New Zealand, Scotland, South Africa and the United Kingdom can be searched simultaneously from this page82. This is not possible anywhere else on the Internet. The databases come from AustLII, BAILII, NZLII, HKLII and SAFLII.
Law Reform Project page on WorldLII
Subject area search example - In similar fashion, special search collections can be created in a particular subject area, rather than for a particular type of source document. An example is the ‘Privacy & FOI’ search option and Project page83, which allows a search over thirteen privacy and FOI tribunals from Australia, Canada, New Zealand, Ireland and Hong Kong, plus the archives (extending back to the mid-90s) of two journals, Epic Alert and PLPR. These databases derive from CanLII, AustLII, BAILII, HKLII, NZLII and WorldLII. A search as extensive as this over privacy-related specialist tribunals is not available anywhere else. In the results below for a search for ‘(health or medical) near disclos*’ the first nine results are each from a different jurisdiction or journal, covering all nine resources currently in the system. More are being added.
Example search from the ‘Privacy and FOI Project page’
There are many other useful selections based on subject matter or document type that are possible from the large and growing collection of databases made accessible by WorldLII. A key element of WorldLII’s development will be to identify areas where numerous collaborating LIIs are providing databases which would be even more valuable if conveniently aggregated across jurisdictions.
3.4.2. More integration: Cross-LII hyperlinks and ‘noteups’
We have seen how WorldLII’s search options allow databases from different LIIs to be combined in a flexible manner. Browsing of documents presents a more challenging task: documents such as cases and articles often cite cases or legislation from other countries, which may in fact be found on another LII. Ideally, the reference should link to the document on the other LII. Ideally, it should also be possible to ‘note up’ a case by selecting a ‘Noteup’ or similar link from the case which finds all cases, articles etc referring to it, irrespective of on which LII the other documents are located: ‘global noteups’.
How can these ‘cross-LII links’ and ‘global noteups’ be automated? Establishing a shared mechanism for automatically recognising the citation of cases from other countries, and whether those cases were located on another LII, so that a hypertext link could be created to that overseas case was identified at the Cornell meeting in 2000 as one of the aims of cooperation between LIIs. This goal can be achieved more easily on centralised systems such as LexisNexis and WestLaw, where all the data is contained on a common technical system and usually in a common format, and with high editorial input. For the free access to law movement, the challenge is to match this value-adding, through cooperation, across a decentralised global collection of LIIs. This requires a number of forms of cooperation:
a consistent, if not identical, case citation method used across LIIs, preferably one that is Court-designated (as discussed below);
comparative citation tables developed by each LII for Courts and Tribunals in their jurisdictions, which map any publisher’s citations to the LII/Court-designated citations
the URLs of cases as they are located on each LII must be able to be inferred from the LII citation of the case, or stated in the comparative citation table;
collaboration in the distribution of these comparative tables in such a way that hypertext mark-up scripts used by LIIs can process these tables to create both the necessary cross-LII links and any ‘global noteups’.
At the LII meeting in Paris in 2003, the LIIs agreed on a method of sharing citation tables, and that each would start developing comparative citation tables for their leading Courts and Tribunals to add to this shared corpus of citations. WorldLII is developing mark-up software which automates the creation of hypertext links from case citations to cases located on any LII, and methods of creating ‘global noteups’. We will now give some details of progress on these issues.
3.4.3. Common standards for citations
From January 1999 the High Court of Australia adopted a new method of 'Court-designated' citation of its decisions; a citation provided by the Court from the moment the decision was made available to the public. That case is Parsons v The Queen [1999] HCA 1 - indicating that it is the first case published by the High Court in 1999. The High Court also adopted paragraph numbering of its decisions (sequential throughout all judgments in a decision), so that paragraph 95 of Parsons has the 'pinpoint citation' Parsons v The Queen [1999] HCA 1 at [95]. This method of citation is sometimes called 'vendor-neutral and medium-neutral', and it is that, but where the Courts themselves add the citation to their decisions the expression 'Court-designated' is preferable as it indicates that it involves the Courts taking control of how their cases are cited.
Since 1966, AustLII had been advocating the adoption of a Court-designated citation by Australian Courts, had proposed a similar method of citation to the Council of Chief Justices in 199884, and had imposed a similar method of citation on many of its databases. Following the lead of the High Court and the Council of Chief Justices, almost all Australian Courts and Tribunals have now adopted the same method of citation, and have adopted an agreed set of Court designators (such as ‘HCA’). It has been implemented on AustLII as the method of citing the decisions of the more than 90 Courts and Tribunals on that system, and is in use by other publishers as a method of parallel citation to their own 'publisher-specific' means of citation. Most Australian Courts and Tribunals have now adopted this method of citation (it is therefore ‘Court designated’). Hence their decisions already have the citation authoritatively added by the Court or Tribunal before the decision is sent to AustLII. Where a Court or Tribunal does not (yet) do this, AustLII adds this form of citation as its own ‘publisher’s citation’.
With the creation of BAILII in 2000, a similar method of citation and set of Court designators was used to identify the decisions of the Courts and Tribunals on that system85. The English Courts, assisted to a very large extent by the efforts of Lord Justice Brooke, quickly took up the challenge of adopting a method of Court-designated citations suitable for electronic publication. The UK Courts adopted the same citation method as used by the Australian Courts, and trialled on BAILII, but with a number of modifications to the Court designators used, implemented as the preferences of the Courts became clear. These changes were then 'retro-fitted' to the BAILII databases. Since then, the English Courts have continued to make advances, as Lord Justice Brooke points out86, by adopting paragraph numbering in their decisions, and through Practice Directions in 200187 and 200288 implementing and consolidating the new system. The first Practice Direction states:
‘The neutral citation will be the official number attributed to the judgment by the court and must always be used on at least one occasion when the judgment is cited in a later judgment. Once the judgment is reported, the neutral citation will appear in front of the familiar citation from the law report series. Thus: Smith v Jones [2001] EWCA Civ 10 at [30]; [2001] QB 124; [2001] 2 All ER 364, etc. The paragraph number must be the number allotted by the court in all future versions of the judgment.’
The second Practice Direction concludes ‘Although the judges cannot dictate the form in which law publishers reproduce the judgments of the court, this form of citation contains the official number given to each judgment which they hope will be reproduced wherever the judgment is republished, in addition to the reference given in any particular series of reports’. By these means, the English Courts have made it clear their intention that the Court-designated citation should be used and not ignored, both by those who are citing cases before a Court, and by those who are re-publishing the Court's decision.
The same citation method has been implemented for the decisions of Courts and Tribunals on PacLII, HKLII, SAFLII and NZLII, though in these jurisdictions these methods of citation have not yet been officially adopted by the Courts. Their status is therefore that of the ‘publishers citations’ of the respective LIIs, but their significance is that their consistent format makes it possible to create new forms of interconnection between LIIs. In Canada a very similar neutral citation method has been adopted by the Courts, and is used on CanLII. For example, the third decision of the Supreme Court of Canada for 2003 has the citation '2003 SCC 3'. This development has had a long gestation in Canada, in which LexUM has been an important participant89.
Consistent citations are therefore now in use in relation to most of the national Courts whose decisions are available through WorldLII. The citations are often Court-designated citations, assigned by the Court before they are sent to the LII concerned, but where they are not they are added by the LII upon receipt.
3.4.4. Cross-LII hyperlinks – Decentralisation + integration
In WorldLII and its collaborating LIIs, cross-LII hypertext links are only implemented to a limited extent as yet. Examples are shown below from HKLII, where the decision in Sin Hoi Chu & others v. The Director of Immigration [2002] HKCFA 390 includes hypertext links to a case on BAILII using the Court-designated citation (EWCA – for the UK Court of Appeal), and to a case on AustLII by use of a parallel publishers’ citation (ie a High Court decision identified by the CLR citation for Commonwealth Law Reports). The same case also includes links to legislation on BAILII (eg at [131], UK Education (Schools) Act 1997).
Example of cross-LII hypertext links in a case on HKLII
Similarly, where any Australian case on AustLII cites a UK case on BAILII by use of its Court-designated citation, there will be a link from AustLII to the case on BAILII. Where cases from Pacific Island countries on PacLII, or South African Courts on SAFLII, cite Australian cases or UK cases by their Court-designated citations, there are links to AustLII or BAILII. Where they cite Australian High Court cases by the CLR citation, such links also appear. However, these examples are still the exception rather than the rule. The LIIs are only now starting to create comparative citation tables, and to use mark-up software which recognises all the citations available on all LIIs. Nevertheless, the process is now underway in a more systematic fashion than before, and advances on one LII in creating comparative citation tables can quickly have a remarkable effect on the quantity of hypertext links between LIIs.
3.4.5. Global ‘Noteups’
Once comparative citation tables are developed, ‘Noteups’ become possible across all LIIs: ‘global noteups’ in the sense of covering everything on WorldLII. As an example of what this promises, selecting the ‘Noteup’ button at the top of the AustLII case Chen Shi Hai [2000] HCA 19 produces the search result shown below. Of the many cases that cite this important Australian immigration decision, six of the first ten search results come from senior United Kingdom Courts, the rest from Australia, as illustrated in part below.
This example creates global noteups from a citation table by searching for all comparative citations for a High Court decision. Alternatively, once the table is used by all LIIs to create hypertext links back to AustLII, only a search for the unique part of the hypertext link pointing to the case ([2000] HCA 19) is needed.
When implemented fully, global noteups will enable a reader of any case to find out not only whether the courts of his or her own jurisdiction have considered that case, but also whether it has been considered in the courts of any other countries involved in WorldLII.
3.5. Strategy 4 – An English+ Interface to LII content
WorldLII is primarily an English language interface to all LII content, and we expect it will stay that way. AustLII/WorldLII is unlikely to have the necessary resources (including multilingual staff) to develop a general multilingual interface. Interfaces in other languages to the whole collection of content shared by the network of LIIs will better emerge elsewhere, and are already doing so for French as Droit Francophone expands its coverage toward being a French language interface to all the LII data. HKLII already has a Chinese language search engine, and could consider developing a Chinese language interface to all content of interest from the LII network, if other databases in Chinese become part of the network. Development of a separate Spanish language hub for LII content is the major omission of the emerging network to date. This illustrates that WorldLII is not in some way the ‘apex’ of all the LIIs, it is rather one possible apex – the one providing primarily an English language interface.
However, because WorldLII becomes involved in some projects that involve aggregating databases of global scope (eg the International Courts and Tribunals, Domain Names, and Privacy & FOI Projects already mentioned), and it is often not possible to place the non-English databases on a more appropriate regional or linguistic hub, WorldLII will inevitably host some databases in a variety of languages. Existing projects require databases in Spanish (International Courts and Tribunals), Khmer (Cambodia) and Portuguese (Timor Leste and International Courts and Tribunals), and the completion of the .ccTLD domain name databases will require databases in numerous languages. In due course we hope that these databases will migrate to other appropriate LIIs. In the meantime, there are steps that we intend to take to make WorldLII more useful in how it deals with these ‘databases of necessity’, while falling short of being genuinely multi-lingual. The proposed improvements are:
We need to improve the SINO search engine so that it properly deals with special characters in more (perhaps all) European languages, enabling more comprehensive and accurate searches in those languages;
Where WorldLII contains a significant number of databases in a particular language (where it is in Roman script), we need to provide a useful database menu and search interface in that language. A prototype of this is the Spanish Project page on WorldLII91 which lists the six databases in Spanish on WorldLII (four international tribunals and two national domain name collections), and a Spanish search interface. The Indonesian Project page provides a search interface in Bahasa92, and numerous Catalog pages in Bahasa. Another example may soon be an interface to Portuguese language databases on WorldLII, if we add more Portuguese language databases than those from Timor-Leste. Other possibilities may emerge93.
Where WorldLII has content in languages using non-Roman characters (eg Cambodian legislation in Khymer) that parallels its English language (or other Roman script databases), we need to make the databases more browseable in those languages, and where possible hypertext linked between equivalent documents. HKLII does this for its equivalent English and Chinese content. Conversion of data into Unicode is being explored to provide a more universal display mechanism for non-European characters.
3.6. Strategy 5 – A Platform for research beyond LIIs
The fifth element that defines WorldLII is that it is intended to be a platform for users to take their research beyond the databases provided by the LIIs. We aim to provide tools that assist comprehensive global legal research. As described in part 2 of this paper, LII content is only one layer of free-access legal content - the layer of consistent highest quality. The tools that WorldLII has developed to go beyond LII content are the WorldLII Catalog and Websearch and ‘Law on Google’.
3.6.1. WorldLII Catalog
The WorldLII Catalog94 (accessible from the bottom left of the front page) may still be the largest law-specific content catalog on the Internet95. The Catalog contains links to over 18,000 websites categorised under more than 6,000 categories with most sites indexed under multiple categories. It lists law-related websites for every country in the world from its ‘Countries’ page, and legally significant international institutions from its ‘International’ page. It has a Subject Index with over 100 subject categories, and other valuable categories such as ‘Law Journals’ and ‘Law Reform’. The Catalog is at present only in English, except for a small part in Bahasa and English. When users search WorldLII, the first three entries in their search results show them the most relevant catalog entries, before the listing of database content, with an option to view more Catalog entries.
The Catalog is maintained by two indexers on WorldLII’s staff, with the assistance of an as yet small number of Contributing Editors96 from around the world. The Catalog has been expanded steadily since 1997, and was funded from1997-2002 by the Asian Development Bank (as part of ‘Project DIAL’97). It is impossible for two people to effectively maintain a global Catalog, even if continuing funding can be found. One alternative for sustainability is to invite participation from international law librarians and other subject or country experts who are willing to take over the development and maintenance of parts of the Catalog. However, both the user interface and the editors’ interface need to be improved before this is likely to be successful.
3.6.2. WorldLII Websearch
The Catalog is the basis for targeting WorldLII’s own web-spider (at those sites that can be spidered), creating a search facility for sites listed in the Catalog ('WorldLII websearch'). This is one of the few law-specific search engines based on web-spidering remote sites98. Droit Francophone has also implemented a similar approach, which now indexes over one million pages99. WorldLII’s websearch spiders are constrained not to ‘wander off’, and will only index sites listed in the Catalog. Its search results ranks sites by conventional word frequency measure, unlike the ‘pagerank’ method used by Google. The combination of the Catalog and websearch allows us to limit the scope of websearch searches to specific areas such as legislation sites, law journals, or international courts and tribunals, which cannot be done with general search engines such as Google100. The choice of the second search option in the example below will limit the websearch to web pages relating to Mongolia (ie from sites listed on any page of the Catalog from this point downwards).
Extract from main Catalog page for Mongolia
We consider that the concept of a websearch targeted by a subject-specific catalog, with websearch results able to be limited by the context of the catalog categories, remains valuable. However, WorldLII’s websearch implementation is deficient in some respects and does not do this model justice: it does not yet index all types of documents (eg PDF files); its search results do not display sufficient contextual information; multiple results from the one site are not concatenated; and it is not well-enough integrated with searches over the WorldLII databases. These deficiencies are being addressed.
Now that general Internet search engines are so broad in their coverage of the web, and so effective in ranking search results, it is a continuing challenge to justify the expense of expanding and maintaining both Catalogs and subject-specific websearch facilities. Our judgment at present is that it is worth doing so, particularly because the WorldLII catalog has pages for every developing county, no matter how small, and these are often the most extensive resource being maintained on the Internet. This is an area where more research is needed on the comparative value of research resources, but for now we will maintain the Catalog and websearch.
3.6.3. ‘Law on Google’
The vast range of content indexed by web spiders for search engines such as Google means that they make searchable many web sites relevant to law that law-specific web spiders such as that operated by WorldLII or Droit Francophone will not reach. Most users do not know how to search Google with any expertise even though they use it frequently. It is too easy to confuse a search that has obtained a large number of ‘hits’ with an effective search. If we can assume that users of WorldLII have a reasonable idea of how to use its search engine, then it becomes valuable to enable them to use the same search syntax to also reach the additional legal materials available via Google, particularly if they can do so by merely repeating a search they have already written in order to search WorldLII .
To accommodate this, WorldLII provides a ‘Law on Google’ option as a ‘repeat this search’ option on each page of search results, and also as a search option on each page of the Catalog. A ‘Law on Google’ search therefore does two things before it sends the user’s search to Google. It translates the user’s SINO search into the syntax required by Google searches, and it attempts to limit the Google search results to law-related materials. For example, a search for ‘(clone or cloning) near human embryo’ will be translated before it is sent to Google into ‘clone OR cloning "human embryo" (law OR legal OR legislation OR regulation OR judgment OR treaty)’. The one aspect of SINO searches that cannot be translated into Google’s syntax is truncation: WorldLII users are asked to repeat any search that contains truncation with the truncation removed before it is sent to Google.
Our assessment is that the ‘Law on Google’ option works very well. We intend to develop it further by attempting to utilise the terms used in defining a user’s location in the Catalog (eg ‘>> Mongolia >> International Trade’) to further constrain a search over Google. This option also needs better integration with searching WorldLII’s databases.
3.6.4. Systematic global legal research
WorldLII provides, however imperfectly, a working model for systematic global legal research over free Internet law resources. Such research involves the following five separate steps, each of which is now possible using WorldLII and its collaborating LIIs:
1 Start research with a search on the most relevant LII (eg PacLII if you live in the Pacific)
2 Expand the search over all cooperating standardised law sites (mainly LIIs) (eg the WorldLII databases, or a subset of them such as ‘All legislation’)
3 Expand the search to non-standardised law sites by a law-specific search engine using web spidering (eg the WorldLII Websearch)
4 Expand the search over a general (ie not law-specific) search engine, but attempt to restrict the results to those that are law-related (eg Law on Google)
5 Browse and search a global catalog of legal web sites to find sites additional content that cannot be found by searching from any central facility (eg sites listed in the WorldLII Catalog)
WorldLII’s interface (and that of some collaborating LIIs) encourages users to take this systematic approach to legal research in three different ways: (i) invitations in search results to repeat searches over different collections; (ii) the WorldLII Catalog interface which provides search options of different scope over different collections; and (iii) all queries search the Catalog entries as well. We have nevertheless concluded that the current interface and options do not make it sufficiently obvious to users what a wide range of options are available to them. Improvement is needed.
Our aim is to assist (and encourage) users to do comprehensive searches over WorldLII Databases, WorldLII Catalog/Websearch and Google without having to re-key searches or learn different search commands. Where a user starts research from a single LII (eg PacLII), search results appear headed by a message such as the following:
Repeat search over WorldLII Databases ; WorldLII Websearch ; Google
The user is therefore invited to broaden their research by repeating the search automatically over WorldLII Databases (represented by the bold lines in the diagram below). When the search is repeated over WorldLII Databases, the search results invite the user to repeat the search over the World Websearch. When the user repeats the search over World Websearch, the search results then invite them to repeat the search over Google.
A systematic research path, starting from a single LII (in bold)
4. Future directions
It is more or less 10 years since the LII (Cornell) and then AustLII started to provide free large-scale Internet access to law. It is only five years since AustLII’s SINO search engine became the main technical device to increase the global spread of free access to law through new LIIs, and only about three years since these LIIs started networking their data, formed the free access to law movement, and endorsed WorldLII as one means of coordinating their efforts. During that time the number of high quality legal databases that have now become accessible on a globally organized basis through WorldLII and Droit Francophone has grown consistently, now amounting to nearly 500 databases, and the LIIs have developed increased technical sophistication, decentralisation, cooperation, and financial sustainability.
These are good indicators for the next 10 years. The free access to law movement is part of the more general movement to develop and protect ‘commons’ or ‘free culture’, most famously exemplified in the free and open source software movements.
Future developments are uncertain, but we have achieved the initial goals that led AustLII to start development of WorldLII: free access to law is no longer an isolated national development, and the feasibility and value of networking and other forms of ongoing collaboration between free access law providers has been demonstrated.
It is now a realistic long-term goal of the free access to law movement to provide an alternative coordinated global means of publication of public legal information, and one which presents its content in ways which are as technically sophisticated as the global commercial legal publishers. To do so it will need to form alliances with many existing organisations that share its goals and include them in its networks, and to assist the establishment of LIIs in more countries. Success will never be a foregone conclusion, but will remain a continuing struggle in all countries to free the law and keep it free.
Bibliography
FN72 - Daniel Adler and The Hon. Sam Sokphal ‘Access to Legal Information in Cambodia: Initial Steps, Future Possibilities’, Proc. 5th Law via Internet Conference, Sydney, 2003, at http://www.austlii.edu.au/au/other/CompLRes/
FN20 - Bruce (2000b) – Bruce, T ‘Tears Shed Over Peer Gynt's Onion: Some Thoughts on the Constitution of Public Legal Information Providers’, 2000 (2) The Journal of Information, Law and Technology (JILT) at http://elj.warwick.ac.uk/jilt/00-2/bruce.html
NQIP - Bruce (2000a) – Bruce, T ‘WORLDLII: A sketch for a distributed search system’, Cornell Law School, 2000 at http://barratry.law.cornell.edu/Summit/worldlii.htm
FN7 - S Burke “Review: World Legal Information Institute” (2003) American Society of International Law (ASIL) Newsletter, January/February 2003 @ http://www.asil.org/newsletterr/wo1031.htm
NQIP - Coulibaly (2003) - Bobson Coulibaly ‘Access to Law in the French-Speaking World: A Renewed Strategy’ [2003] CompLRes 2 at http://www.austlii.edu.au/au/other/CompLRes/2003/2.html
FN26 - Greenleaf (2000) - G Greenleaf ‘Free the Law: How the Australasian Legal Information Institute (AustLII) Achieved the Free Availability of Legal Information on the Internet’ 2000 (1). The Journal of Information, Law and Technology (JILT) at http://www2.warwick.ac.uk/fac/soc/law/elj/jilt/2004_2/greenleafmowbrayandchung
FN26 (29) - Greenleaf (2004) - Greenleaf G ‘Jon Bing and the History of Computerised Legal Research – Some Missing Links’ in Olav Torvund and Lee Bygrave (Eds) Et tilbakeblikk på fremtiden ("Looking back at the future") 61-75, Unipub, Oslo, 2004
FN66 Greenleaf (2004a) – ‘Full free access to law: Global policy aspects’ (Powerpoint presentation) 6th Law via Internet Conference, Paris, November 2004, available at http://www2.austlii.edu.au/~graham/publications/2004/Policies_Paris.ppt
FN49 - Greenleaf (1998) - Greenleaf G, (1998) Developing the Internet for Asian Law- Project DIAL (A feasibility study and prototype) (commissioned Report to the Asian Development Bank), 156 pgs http://www2.austlii.edu.au/~graham/DIAL_Report/Report.html
FN49 - Greenleaf, Chung and Allen (2002) - Graham Greenleaf, Philip Chung and Russell Allen 'World Law: Finding law after Google' (2002) UTS Law Review, Vol 4, pgs 11-30, Halstead Press, Sydney, ISSN 1442 4959; also in Proc. AustLII Law via Internet 2001 Conference, AustLII, Sydney, 2001
FN5 - Greenleaf, Chung and Mowbray (2004) - Greenleaf G, Chung P and Mowbray A ‘A new home online for Commonwealth law: A proposal for a CommonLII’ Journal of Law Information and Technology (JILT), 2004
Greenleaf, Chung and Mowbray (2005) – Greenleaf G, Chung P, Mowbray A “Responding to the fragmentation of international law - WorldLII’s International Courts & Tribunals Project” Canadian Law Library Review (in publication, accepted January 2005)
FN 35 - Greenleaf, Chung et ors (2002) - G Greenleaf, P Chung, A Mowbray, Ka Po Chow and KH Pun 'The Hong Kong Legal Information Institute (HKLII): Its role in free access to global law via the Internet [2002] Hong Kong Law Journal Vol 32, Part 1; draft presented at 4th Law via Internet Conference, Montreal, October 2002, available at http://www.lexum.umontreal.ca/conf2002/actes/greenleaf.html
FN49 - Greenleaf et al (2000) Greenleaf G, Austin D, Chung P, Mowbray A, Matthews J and Davis M Solving the Problems of Finding Law on the Web: World Law and DIAL', 2000 (1) The Journal of Information, Law and Technology (JILT) ; also in Proceedings, 16th Biennial LAWASIA Conference, Seoul, Korea, 7-11 September 1999
FN26 - Greenleaf, Mowbray and King (1997) - G Greenleaf, A Mowbray G King ‘New directions in law via the internet - The AustLII Papers’ Journal of Information, Law and Technology (JILT), Issue 2, 1997, University of Warwick Faculty of Law, at http://elj.warwick.ac.uk/jilt/issue/1997_2
FN26 - Greenleaf, Mowbray, King and van Dijk (1995) - Greenleaf, G, Mowbray, A. King, G and van Dijk, P (1995) ‘Public access to law via internet: the Australasian Legal Information Institute’ (1995) Journal of Law & Information Science, Vol 6 Issue 1
FN86 Lord Brooke (2003)- Lord Justice Henry Brooke ‘Publishing the Courts: Judgments and public information on the Internet’ Proc. Commonwealth Legal Conference 2003
FN1 - Lord Cooke of Thorndon “The Dream of an International Common Law” in C Saunders (Ed) Courts of Final Jurisdiction: The Mason court in Australia, Federation Press 1996.
NQIP - Lord Cooke (1997) - Lord Cooke of Thorndon ‘The Judge in an Evolving Society’, address to Judges and Judicial Officers of the High Court of Hong Kong, 17 December 1997, available at http://www.upf.pf/recherche/fichiers%2520RJP4/01Cooke.doc&e=673
FN84 - Mowbray, Greenleaf and Chung (2000) - A Mowbray, G Greenleaf and P Chung 'A Uniform Approach for Vendor and Media Neutral Citation - the Australian Experience' Citations Workshop: strategies for accessing law and legal information Edinburgh, Scotland - 11th & 12th March 2000
FN54 - Mowbray, Austin and Chung (2000) - Mowbray A, Austin, D and Chung, P ‘Scalability of Web Resources for Law: AustLII's Technical Roadmap: Past, Present and Future’, The Journal of Information, Law and Technology (JILT), vol 1 2000
FN65 - Poulin and Perpignand (2001) – Daniel Poulin and Ernst Perpignand ‘Canlii 2000-2004 - Technical Strategy’ [2001] CompLRes 2 at http://www.austlii.edu.au/au/other/CompLRes/2001/2.html; 3rd Law via Internet Conference, Sydney 2001
FN1 - D Poulin “Open Access to Law in Developing Countries” First Monday Volume 9, Number 12, 6 December 2004 http://www.firstmonday.org/issues/issue9_12/poulin/index.html (originally presented at the 5th Law via Internet Conference, AustLII, Sydney, November 2003)
FN35 - Pun et ors (2003) - Pun, K H, Chong, C,F, Chan, Vivien ‘Processing Legal Documents in the Chinese-Speaking World’ [2003] CompLRes 4 at http://www.austlii.edu.au/au/other/CompLRes/2003/4.html
FN58 - Wittfoth et al (2005) - Armin Wittfoth, Philip Chung, Graham Greenleaf and Andrew Mowbray ‘AustLII’s Point-in-Time legislation system: A generic PiT system for presenting legislation’, paper distributed at public launch of the Point-in-Time legislation system, 7 April 2005, available at http://portsea.austlii.edu.au/pit/
FN58 - Wittfoth et al (2003) - Armin Wittfoth, Philip Chung, Andrew Mowbray, and Graham Greenleaf (2003) ‘Can One Size Fit All?: - AustLII’s Point-in-Time Legislation Project’, 5th Law via Internet Conference, Sydney, 2003
FN58 - Wittfoth et al (2001) - Armin Wittfoth, Philip Chung, Andrew Mowbray, and Graham Greenleaf (2001) ‘Towards a Uniform Representation of Multi-jurisdictional Legislation-like Instruments’, 3rd Law via Internet Conference, Sydney, 2001
NQIP - Yamase (2003) – The Hon. Dennis K Yamase ‘The Law of the Federated States of Micronesia’ [2003] CompLRes 7 at http://www.austlii.edu.au/au/other/CompLRes/2003/7.html
* Graham Greenleaf is Professor of Law, University of New South Wales; Philip Chung is Executive Director of AustLII and WorldLII and Lecturer in Law, University of Technology, Sydney; Andrew Mowbray is Professor of Law and Associate Dean, Faculty of Law, Universtity of Technology, Sydney. This article covers the development of WorldLII to mid-2005. A further article covering WorldLII's development from 2005-08 will be included in SCRIPT-ed later in 2008. The delay in publication results from closure of another journal, and the authors wish to acknowledge SCRIPT-ed's rapid publication upon receipt of this manuscript.
1 See Lord Cooke of Thorndon “The Dream of an International Common Law” in C Saunders (Ed) Courts of Final Jurisdiction: The Mason court in Australia, Federation Press 1996; D Poulin “Open Access to Law in Developing Countries” First Monday Volume 9, Number 12, 6 December 2004 @ http://www.firstmonday.org/issues/issue9_12/poulin/index.html (originally presented at the 5th Law via Internet Conference, AustLII, Sydney, November 2003).
2 See lists of acquisitions prepared by the American Association of Law Libraries at http://www.aallnet.org/committee/criv/resources/tools/list/reedelsevier.html
3 We will not discuss in detail here why should we value free access to legal information. In relation to the laws of one own country, the most obvious reason is that access to legal information supports the rule of law. People should not be governed by laws to which they do not have effective access. Businesses have much the same needs as individuals. Also, attraction to foreign investment is enhanced by free access to information about the operation of a country's legal system. Transparency of a country's legal system is one of the three legal and administrative requirements for WTO membership, and this is enhanced by free access to at least regulatory materials. We also value free access to the laws of other countries, for the reasons sketched above.
4 This Declaration is discussed in the next section.
5 Greenleaf, Chung and Mowbray (2004) G Greenleaf, P Chung and A Mowbray “A new home online for Commonwealth law: A proposal for a CommonLII” Journal of Law Information and Technology (JILT) (2004) @ http://www2.warwick.ac.uk/fac/soc/law/elj/jilt/2004_2/greenleafmowbrayandchung/
6 See Poulin, note 1, for a detailed discussion of technical and administrative means by which this can be done; See also Bing (2003).
7 See S Burke “Review: World Legal Information Institute” (2003) American Society of International Law (ASIL) Newsletter, January/February 2003 @ http://www.asil.org/newsletterr/wo1031.htm : ‘Though it is only a few months old, WorldLII is certainly a significant step toward achieving the goals set forth in the Montreal Declaration, holding its own against Westlaw and LexisNexis.’
8 Another answer to providing comprehensive access is to have all significant legal websites in a jurisdiction adopt a standard format for their materials, and to return results of distributed searches to a centralised search facility. There are no successful examples of this approach yet known.
9 We are not suggesting that LIIs should only provide essential legal information. They are likely to be involved in the provision of other types of secondary materials such as law journals, in the provision of 'plain English' guides to the law and in other approaches to improving public access to the law. These sources require different considerations from 'essential' legal information, particularly because their publication is less likely to be pursuant to a duty to publish, or public subsidies to do so.
10 The definition is useful, but may ultimately be too limiting, as it would, for example, exclude large free-access collections of secondary legal information (journal articles) produced in publicly-funded institutions. The Legal Scholarship Network has the same motivations and value as any LII.
11 There is a list of LIIs @ http://www.worldlii.org/catalog/52806.html
12 See http://www.juridicas.unam.mx/infjur/leg/ - It is maintained by the Instituto de Investigaciones Jurídicas (Legal Research Institute) de la UNAM (Universidad Nacional Autónoma de México).
13 See http://zamlii.zamnet.zm/ - It seems to have been revived in 2004.
15 Links to these key government sites, and to LIIs, can be found @ http://www.worldlii.org/catalog/52806.html
16 See http://www.worldlii.org/links/2172.html for a global list.
17 See http://www.worldlii.org/links/2027.html for a global list.
18 D Poulin, note 1.
20 See T Bruce, “Tears Shed Over Peer Gynt's Onion: Some Thoughts on the Constitution of Public Legal Information Providers” (2000) (2) The Journal of Information, Law and Technology (JILT) @ http://www2.warwick.ac.uk/fac/soc/law/elj/jilt/2000_2/bruce
21 Australia’s aid agency, AusAid, funded the Cambodian participation.
22 ADIJ, ADBS and Juriconnexion: for details see http://www.frlii.org/article.php3?id_article=14
23 The famous maxim of Internet governance adopted by the Internet Engineering Task Force (IETF).
25 The Sydney amendments were:.(i) in the title of the Declaration, ‘public’ was changed to ‘free’; (ii) the words ‘where possible’ were deleted from the second bullet point ‘where possible, free of charge’; (iii) addition of the description of a legal information institute and the encouragement to participate in networks; and (iv) addition of the final bullet point about an annual meeting to the list of areas of agreed cooperation. The Paris amendments were: (i) Addition of ‘It also includes legal documents created as a result of public funding’ and (ii) addition of the bullet point concerning conditions of re-use.
26 For various versions over time see G Greenleaf, A Mowbray, G King, and P van Dijk, (1995) “Public access to law via internet: the Australasian Legal Information Institute” (1995) Journal of Law & Information Science, Vol 6 Issue 1; G Greenleaf, A Mowbray and G King “New directions in law via the internet - The AustLII Papers” (1997) (2) Journal of Information, Law and Technology (JILT), University of Warwick Faculty of Law, @ http://www2.warwick.ac.uk/fac/soc/law/elj/jilt/1997_2/greenleaf; G Greenleaf, “Free the Law: How the Australasian Legal Information Institute (AustLII) Achieved the Free Availability of Legal Information on the Internet” (2000) (1) The Journal of Information, Law and Technology (JILT) @ http://www2.warwick.ac.uk/fac/soc/law/elj/jilt/2000_1/free_the_law/transcript and G Greenleaf, “Jon Bing and the History of Computerised Legal Research – Some Missing Links” in Olav Torvund and Lee Bygrave (Eds) Et tilbakeblikk på fremtiden ("Looking back at the future") 61-75, Unipub, Oslo, 2004.
27 Poulin, note 1, takes a similar position: ‘In order to be truly open, publication must not be restricted by any conditions as to subsequent use or republication of the documents, except for conditions needed to ensure accuracy.’
28 This seventh element was not included in our original list in 1995, but the importance of citations has subsequently become more obvious.
29 See Greenleaf, note 26, for a more detailed discussion as part of a critique of Jon Bing’s views on developing national Legal Information Systems
30 Exclusion of general search engine robots from case law (for privacy reasons) and from legislation (for reasons of affect on performance and lack of comprehensiveness) is common practice by LIIs (eg AustLII and CanLII).
31 Robot Exclusion Protocol; for current and proposed versions see the Robots Exclusion pages @ http://www.robotstxt.org/wc/exclusion.html
32 An example is most of the content on the US Library of Congress GLIN website.
33 They are also an example of category 4, but deserve special treatment for our purposes.
34 For examples of discussions see UC Berkeley Library @ http://www.lib.berkeley.edu/TeachingLib/Guides/Internet/InvisibleWeb.html; Wikipedia entry for ‘Deep Web’ @ http://en.wikipedia.org/wiki/Hidden_web
35 See for example the difficulties of building Chinese language legal search engines: G Greenleaf, P Chung, A Mowbray, Ka Po Chow and KH Pun “The Hong Kong Legal Information Institute (HKLII): Its role in free access to global law via the Internet” [2002] Hong Kong Law Journal Vol 32, Part 1; draft presented at 4th Law via Internet Conference, Montreal, October 2002, available @ http://www.lexum.umontreal.ca/conf2002/actes/greenleaf.html and KH Pun, CF Chong, Chan, Vivien “Processing Legal Documents in the Chinese-Speaking World” [2003] CompLRes 4 @ http://www.austlii.edu.au/au/other/CompLRes/2003/4.html
36 The decline of interest in LegalXML over the past three years seems to be an example of this.
37 WorldLII does this to obtain most of the data in its International Courts and Tribunals Project (discussed later), extracting data from many Court and Tribunal sites where no automated extraction processes are possible, and obtaining date in file formats such as PDF which require considerable conversion into HTML.
38 AustLII does this to obtain results from CCH for its publishers search, and WorldLII does so to obtain CanLII results.
39 WorldLII Catalog: http://www.worldlii.org/catalog/270.html
40 The Droit Francophone catalog is presented together with the databases for each country: http://portail.droit.francophonie.org/
41 These include CCH Australia and Australian Business Limited, an industrial law publisher.
42 About half the case law databases on AustLII (46/90) are the only online collections for the respective Courts or Tribunals: see details @ http://www.austlii.edu.au/cgi-bin/cases_status.cgi
43 As Poulin, note 1, puts it ‘decentralised actions are the most viable means of achieving results’
44 The LII (Cornell) could become a US hub for federal law, if a way is found to include its distributed access to the various US Courts of Appeal decisions into the network.
45 PacLII is essentially a very extensive regional aggregator, but is also to some extent a hub in relation to its incorporation of the content of other small LIIs in the Pacific.
46 As Poulin, note 1, says ‘it seems improbable that global publication of all legal content on the web will result from the emergence of a sole mega-website’ – but it is something of a straw man as no one proposes this.
47 Poulin, note 1, puts this well: ‘If such standards are prepared, nothing will hinder the harmonious development of local initiatives while building a global and comprehensive access to the entirety of the legal Internet.’
48 Poulin, note 1.
49 See G Greenleaf, (1998) Developing the Internet for Asian Law - Project DIAL (A feasibility study and prototype) (commissioned Report to the Asian Development Bank), 156 pgs http://www2.austlii.edu.au/~graham/DIAL_Report/Report.html; G Greenleaf, D Austin, P Chung, A Mowbray, J Matthews and M Davis “Solving the Problems of Finding Law on the Web: World Law and DIAL',” (2000) (1) The Journal of Information, Law and Technology (JILT) @ http://www2.warwick.ac.uk/fac/soc/law/elj/jilt/2000_1/greenleaf; also in Proceedings, 16th Biennial LAWASIA Conference, Seoul, Korea, 7-11 September 1999; G Greenleaf, P Chung and R Allen “World Law: Finding law after Google” (2002) UTS Law Review, Vol 4, pgs 11-30, Halstead Press, Sydney, ISSN 1442 4959; also in Proc. AustLII Law via Internet 2001 Conference, AustLII, Sydney, 2001.
50 Poulin, note 1.
51 The rest of AustLII’s funding comes (in decreasing order) from our two host Universities, from the legal publisher CCH (because of AustLII’s ‘Publishers Search’) facility, from a business group, from a small number of government agencies and Courts whose data AustLII publishes (cases, treaties, law reform reports etc), and from some law firms and bar organisations.
52 Andrew Mowbray is the principal author of the main AustLII software, particularly the SINO search engine. Other AustLII staff have made major contributions to AustLII’s set of mark-up tools, including Philip Chung, Joseph Kwok, Armin Witforth, Trevor Roydhouse and (before 1998) Geoffrey King and Peter van Dijk. The cataloguing /websearch software is by Daniel Austin with contributions from Russell Allen and Philip Chung.
53 ‘Size Is No Object’ – An offhand description of the fact that it scales up well, but the name has stuck.
54 For a description, see A Mowbray, D Austin, and P Chung, “Scalability of Web Resources for Law: AustLII's Technical Roadmap: Past, Present and Future” (2000) 1 The Journal of Information, Law and Technology (JILT) @ http://www2.warwick.ac.uk/fac/soc/law/elj/jilt/2000_1/austin
55 This is similar to one of the ideas behind Google’s PageRank, but over a much more controlled data set.
56 See AutoSearch notes: http://beta.austlii.edu.au/austlii/help/autosearch.html
57 See Mowbray, Austin and Chung, note 54.
58 See AustLII’s Point-in-Time Project @ http://portsea.austlii.edu.au/pit/ and A Wittfoth, P Chung, A Mowbray, and G Greenleaf (2001) “Towards a Uniform Representation of Multi-jurisdictional Legislation-like Instruments”, 3rd Law via Internet Conference, Sydney, 2001; A Wittfoth, P Chung, A Mowbray and G Greenleaf (2003) “Can One Size Fit All?: - AustLII’s Point-in-Time Legislation Project”, 5th Law via Internet Conference, Sydney, 2003; A Wittfoth, P Chung, G Greenleaf and A Mowbray (2005) “AustLII’s Point-in-Time legislation system: A generic PiT system for presenting legislation”, paper distributed at public launch of the Point-in-Time legislation system, 7 April 2005, available @ http://portsea.austlii.edu.au/pit/
59 See Mowbray, Austin and Chung, note 54.
60 CanLII uses the Eliisa search tool developed by LexUM using the Jakarta Lucene library; see http://www.canlii.org/aide-help_en.html#syntax for a summary and http://www.canlii.org/reference_guide.pdf for the Reference Guide
61 These criteria were presented, in slightly simpler forms, first at the ‘Free the Law’ Seminar in London, November 1999, and then at the 4th Law via Internet Conference in Montreal, October 2002.
62 This includes no condition of identification of users (eg password-based logins), and consequently no surveillance of users.
63 This includes, in the case of government bodies, not doing anything to undermine independent publishers from obtaining access to public legal information for the purposes of republication. This is an anti-monopoly requirement.
64 See Poulin (2004), note 1.
65 See D Poulin and E Perpignand “Canlii 2000-2004 - Technical Strategy” [2001] CompLRes 2 at http://www.austlii.edu.au/au/other/CompLRes/2001/2.html; 3rd Law via Internet Conference, Sydney 2001; and Poulin, note 1, particularly ‘Software and Tools’
66 See G Greenleaf, “Full free access to law: Global policy aspects” (2004) (Powerpoint presentation) 6th Law via Internet Conference, Paris, November 2004, available @ http://www2.austlii.edu.au/~graham/publications/2004/Policies_Paris.ppt
68 Two databases from Interrights consisting of summaries of major national decisions from many countries concerning human rights issues are also included.
69 See http://www.worldlii.org/hosted_databases.html#int
70 Poulin, note 1.
71 See generally on these developments Poulin, note 1.
72 See D Adler and The Hon. Sam Sokphal “Access to Legal Information in Cambodia: Initial Steps, Future Possibilities”, Proc. 5th Law via Internet Conference, Sydney, 2003, @ http://www.austlii.edu.au/au/other/CompLRes/
77 Poulin, note 1.
78 See http://www.worldlii.org/au/other/CompLRes/; only papers from the 1997, 1999, 2001 and 2003 Conferences are available as yet
79 CanLII Technical Library @ http://www.canlii.org/tech_en.html
81 See http://www.worldlii.org/forms/search1.html; the user holds down the Ctrl key and selects databases from the full list.
82 In some cases, as with the Australian Law Reform Commission, the collections are complete (back to 1975), whereas in other cases the relevant LIIs only have current reports, but are starting to capture the backset.
84 For a history of these developments in Australia and elsewhere, see A Mowbray, G Greenleaf and P Chung “A Uniform Approach for Vendor and Media Neutral Citation - the Australian Experience” Citations Workshop: strategies for accessing law and legal information Edinburgh, Scotland - 11th & 12th March 2000.
85 See Mowbray, Greenleaf and Chung, note 84, Part 8 for discussion of the implementation on BAILII and the original set of designators used.
86 See Lord Justice Henry Brooke, “Publishing the Courts: Judgments and public information on the Internet” Proc. Commonwealth Legal Conference 2003, particularly 'Advances in England and Wales in the last five years'.
87 Practice Direction (Judgments: Form and Citation) [2001] 1 WLR 194 available @ http://www.lawreports.co.uk/civjan0.3.htm
88 Practice Direction (Judgments: Neutral Citation) [2002] 1 WLR 3 available @ http://www.lawreports.co.uk/othjanb0.1.htm
89 See ‘RefLex: CanLII's Citation Resolver’ @ http://www.canlii.org/reflex_en.html
91 See http://www.worldlii.org/int/special/espanol/; Proyecto de Base de Datos en Español.
92 See http://www.worldlii.org/id/index_id.html; Proyek Database Hukum Indonesia.
93 It might also be useful at present to have a French language interface in order to better search French language content from WorldLII on Droit Francophone, CanLII, decisions of international Tribunals, and the Cambodian databases on WorldLII, but in due course all this content, and much more, is likely be available via Droit Francophone, and it would be better to leave the francophone interface to LII content to them.
94 See http://www.worldlii.org/catalog/270.html for the opening ‘Categories’ page.
95 For a detailed description, see Greenleaf, Chung and Allen (2002). The word ‘content’ is used to exclude (for purposes of comparison) the thousands of links to private investigators and law firms found in the law sections of the Google Directory / Open Directory Project (OPD).
97 See http://www.worldlii.org/catalog/dial/ for the DIAL (‘Development of the Internet for Asian Law’ archive).
98 Other than Droit Francophone’s similar facility, we are unaware of any other significant current examples.
100 For example, searches over Law section of the Google Directory / Open Directory Project (ODP) http://directory.google.com/Top/Society/Law/ only search the catalog entries, and even though Google may hold the full texts of all or most of the catalog entries, it is not possible to limit a Google web search to only those sites listed on a particular page of their catalog.