• Home
• Research
• Software
• Facilities
• People
• Publications
• Contact us

The intellectual property economy, and, more narrowly, the patent economy, form an incredibly wide-reaching and important part of the economic activity of the United States, and of the broader world. The patent ecosystem involves a diverse collection of players and interests, as can be seen by following the lifecycle of a typical patent in Fig. 1. Inventors conceive of an idea, then patent agents and attorneys help them author and defend/refine a patent application, interact with patent examiners at the patent offices who review it, and check for novelty, prior art, and usefulness (or industrial applicability). Once the patent is issued, and even sometimes while it is still in the application phase, middlemen companies may buy and sell it. A company owning a patent may then request that other companies license its use. Standards organizations, such as 3GPP or IEEE, are also important participants in the lifecycle of many important patents, since often certain patents are required in order to implement standards, and hence license agreement structures, such as the FRAND (fair reasonable and non-discriminatory) agreement, are set up to coordinate their licensing for use. Finally, if a company decides that another company is likely infringing on its patent, it may bring a case, which the federal courts must then hear.

To achieve these goals, these participants perform multiple time-consuming and costly methods as depicted in Fig. 1. Inventers and patent agents must train and practice in the art of claim writing in order to write proper claims [1]. These claims need to cover multiple variations of an inventer’s idea but not be too broad to cover already created inventions. Prior art search, used when applying for a patent as well as in litigation, can require hundreds of man-hours to find just a handful of related documents from a pool of potentially millions of unrelated documents [2, 3, 4, 5]. Patent examiners need to understand the patent classification systems they use as well as the different categories in their field in order to determine the correct categories to assign a patent [5]. When companies come together to write standards, these companies need to determine which of their patents are related to the standards documentations. If a particular company did not suggest that particular addition to the standard, they will need to search their patent profile to see if any patents could be related to that addition[6, 7, 8, 9, 10]. Patent valuation requires analyzing trends in the technology to predict the value of a particular technology via reading and analyzing hundreds of documents about technology [11, 12, 13, 14, 15, 16].

Most of these tasks require determining a way of representing whole patents for comparison against other patents and even other non-patent documents. One method that preserves the grammatical structure of a patent is a dependency list [17, 18]. Dependencies list pair of words in a sentence and explain how they are related. Dependency relationships can be generated from parse trees - ordered, rooted trees that represent the syntactic structure of sentences.

Software packages that are used to generate dependencies from sentences such as Stanford’s Dependency Software are trained typically on a corpus of written text from newspaper articles such as the Wall Street Journal. Patent claims, which must combine a mix of legal and technical terms, and must fit the entire legal scope of a part of an invention in one sentence, use grammar and language not typically found in newspaper articles. As [19] and [20] discuss and as can be observed in Fig. 2, this can adversly affect the way NLP parsers work with patent claims.

One way to fix the dependencies was discovered by noting that some words are given the wrong parts of speech tag. By forcing these incorrect parts of speech tags (initially generated by Stanford’s POS Tagger) to be correct, the dependencies can be made correct. We had noticed before that most of the problems in the parsing of the sentence came from words that the parser tagged incorrectly as verbs. By focusing on just correctly re-tagging words declared to be verbs, a much more accurate parse and set of dependencies could be developed.

An automated system that will correct parts of speech tags can be created to correct this issue but data of incorrect POS tags and correct POS tags needed to be collected first. An Amazon Mechanical Turk campaign was run to gather our data. This campaign provided workers with segments of patent claims and asked them what parts-of-speech tag each “verb” actually is. Data we deemed acceptable (given to us by those that correctly completed randomly inserted test questions) was then used as training data for our automatic correct.

The automatic corrector for patent claims that we developed follows a training stage as shown in Fig. 3 and a testing stage as shown in Fig. 4. To perform feature extraction in this model, we gather the verb-in-question as well as both the word that immediately precedes it as well as the word that immediately follows it. We generate word vectors of these three words using word2vec in order to create a vector that not only accurately and uniformly represents the words as vectors but also allows similar sets of words to be closer than dissimilar sets of words.

We noticed when investigating this problem by hand initially that some words that were tagged as a verb are always another specific tag. For instance, the word “said” is always used as an adjective (e.g. “said motorized vehicle” where the claim is referencing the motorized vehicle that was discussed earlier) even though it is almost always declared to be a verb by the POS tagger. The rule-based corrector is just a set of ever-expanding and specific rules developed to automatically change and set the POS tags for those words that we noticed should always be a particular tag.

We have confirmed that the automatic tag corrector allows for the creation of parsed patent claims that more accurately reflect the true parse of the patent claims. Using parse trees generated from the AMT data as the gold standard, we compared the parse trees from generated straight from the Stanford Parser and the trees generated after using our automatic tag corrector. No matter how we split which portion of the data was the training data and which portion was the testing data, the parse trees generated after using our automatic corrector were closer to the AMT results than the plain Stanford Parser parse trees.

The testing stage is performed multiple times after the initial correcting. This is done because, sometimes, after rerunning the POS tagger, different words are incorrectly labeled as verbs. During each round these new verbs are rerun to determine their correct POS tags before these corrected tags (as well as the corrected tags from the previous rounds) are fed back into the system until no new verbs are obtained.

After we have gathered our data, one area of interest that we hope to use it is in the field of patent classification. Our initial experiments involved determining the best features to use in an SVM-based classification system for the USPC classification system.

We performed an initial test run of future classification experiments using just two classes: 714/748 and 714/763 of the USPC system. These two classes fall under the main class of “Error Detection/Correction and Fault Detection/Recovery”. Class 714/748 is labeled as “Request for Retransmission” which itself is a subclass to Class 714/746 (“Digital Data Error Correction”) which itself is also a subclass to Class 714/699 (“Pulse or Data Error Handling”). Class 714/763 is labeled as “Memory Access”. This class is a subclass of Class 714/752 (“Forward Correction by Block Code”) which is a subclass of Class 714/746 which is a subclass of Class 714/699.

For the experiments, we ran 5 different systems as well as combinations of those five. We generated basic tfidfs over the trigrams of both the claims (claim_tfidf) and the abstract (abstract_tfidf). We also generated dependencies over both of them (claim_origDep and abstract_dep). Finally we fixed the dependencies of the claims and rerun them to generate fixed dependencies (claim_fixDep). Classification was done using a simple SVM.

While neither of the two claim dependencies features outperformed the basic trigram features of the claims, they did provide improvement when we combined them. In addition, by fixing the dependencies, we obtain a much better performance than using the original original. As the system learns more about the text, both in physical (trigrams) and gramatical (dependencies) structure, the performance improves.

More complicated experiments in this field will be run to determine just how much improvement is possible by using different features.

Similar to the previous topic, another topic we are interested in is identifying patents related to particular standards technologies via a similar classification system. A key issue that is of increasing importance to the development of industrial technology standards such as communications standards has been potential infringement on existing patents, and the resulting necessary licensing required to implement the standard. This issue is of importance to nearly all of the players involved in the associated patent economy. Traditionally, this issue has been handled via IPR disclosure mechanisms, however, this leaves out consideration of patent holders not actively participating in the standardization process but holding patents that are infringed upon by the standard. Were standards organizations to know, at a low cost, that such patents existed, they could potentially avoid them by standardizing around them. On the other side, IP middlemen would like to know about these patents, as these form interesting patents to buy. The government would like to be able to detect this as well, as it could form a key issue when examining patents at the USPTO, as well as in reducing the burden on the Federal courts with the associated cases. Additionally, the IPR disclosure process can take a long time and standards organizations involve competing businesses, so low-cost mechanisms of tracking what competitors are filing that are essential for the standards would be desirable as well. We are designing algorithms that will detect whether or not a patent is essential for a standard. These algorithms could provide powerful tools for the monitoring and evaluation of patents for a wide variety of stakeholders.

Presently, this is a labor intensive task requiring tedious and thorough review of large documents by both legal and engineering/scientific experts, using years of training and hours of effort to register vague, arcane, or outdated language and terminology against often newly minted vocabulary buried within gigantic unwieldy documents. Determining which patents are essential for implementing parts of a standard is a key high value problem for many players in the intellectual property economy, and hence the availability of a recommender system to speed the retrieval and registration work would be widely appreciated.

In addition to gathering training and evaluation “ground-truth” data regarding these patent to standards maps to be carried out as part of data-collecting campaign, we will expend substantial effort developing information retrieval systems oriented toward mapping patent claim language to language in standards. A key component of this work will be the adaptation of NLP tools to the language found in patent claims and standards enabled through the data collected previously, which will enable us to separate the underlying meaning of claims and standards from the jargon and language patterns they must use.

Fortunately, the existing base of literature on cross document class information retrieval is expected to be useful for the patent to standard mapping task, and many of these also have integrated existing NLP tools. Most likely, the system will build off what appears to be a developing consensus in the design of patent information retrieval systems. In particular designs appear to have converged to performing query expansion [21] informed with pseudo relevance feedback [22, 23, 24], could be informed by ontologies [25, 26, 27, 28, 29], and will probably need to be informed by sources such as wikipedia [30] so as to recognize variants of the same terms. Additionally, the design may potentially generate multiple queries [31], and will likely integrate (automatically classified [32, 33, 34, 35, 36]) patent subject codes, as well as syntax structures [37] learned through NLP, in the retrieval and ranking [38].

Another key novel aspect in our approach will be the integration of online and active learning into the recommendation system to enable it to continue to learn online as trained users interact with it. When a recommended map between a patent claim and the standard is accepted and used by the patent professional using the system, this provides a new bit of training information to enable the classifiers embedded into the retrieval system to further adapt. When it is rejected and another one is entered into the system by the expert, this too provides a new bit of training information with which to adapt the classifiers. Additional to this online adaptation, active learning [39, 40] can be utilized during the initial phase of hand mapping patents with claims known to be essential to parts of a standard. The purpose of integrating active learning at this stage is to select which of these patents to map to standards with highest priority by selecting those that appear to be the most informative for further training the classifier. By integrating the training of the classifier into the labeling process and selecting the data points that are most informative to label, the amount of data that will be required to be labeled to train the classifier to a given performance will be decreased relative to a technique which labels the data blindly of the classifier.

References