本文へ移動

Hikidashi引き出し

和暦と漢数字の相互変換、活用した語の辞書形、漢字への読みの割り当て、貼り付けた文からの単語の抽出、文の難しさ。日本語のテキストを扱う小さな道具を集めた引き出しです。

「引き出し」という名前は、道具が一つずつしまってあり、必要なときに取り出して使える、という意味です。 名前について(英語)

目次

リファレンス

貼り付けた文

import { … } from "@johnmorrisdotca/hikidashi/extract";

最小化と gzip 圧縮をしたサイズは 2.2 kB(読み込むものをすべて含み、esbuild で計測)

リファレンスの説明と例は英語です。見出しと目次は日本語です。

抽出: Japanese words and kanji pulled out of pasted text.

Somebody pastes a page of a book, a lesson handout, a chat message, and wants the kanji and the words out of it. Two things follow from "somebody pastes anything".

The first is safety. The text is never stored, never rendered as anything but text, and never trusted for its length: it is cut to a cap before any work is done, control characters are dropped, and what comes back is not the text but words your own dictionary recognised and single kanji, a fixed shape of strings. Nothing a paste can contain survives that as anything but a kanji or a word you already knew.

The second is the reading itself: the runs of Japanese are found, the words in them are matched longest-first against the dictionary you pass in as known, and every kanji is taken whether it belongs to a matched word or not. A verb or adjective is matched as it is written (行きます, 高くて) and returned as the dictionary form known lists (deinflect), so a sentence gives 行く rather than the noun 行き that its stem happens to spell. The package carries no dictionary of its own: known is a Map from each word to the kinds of word it is, from yours.

エクスポート

名前種類概要
EXTRACT_LIMITS定数What a paste is held to: read to 20,000 characters and then cut, words of up to 8 characters, and at most 4,000 candidates asked of a dictionary.
extractFromText関数The words and kanji a paste comes to: the words known lists, longest match first, and every kanji in the text.
ExtractInput型What to read, and the dictionary to read it against: the argument of extractFromText.
ExtractResult型The words found, then the kanji, each once, in the order they first appear; and what the paste held.
KnownWords型The words your dictionary confirmed, each with the kinds of word it is.
sanitizePastedText関数The text as it will be read: control and invisible characters (and line and paragraph separators) turned into spaces, then cut to EXTRACT_LIMITS.characters.
wordCandidates関数Every substring the dictionary might know, so one question can be asked about the whole paste: look all of them up at once, and pass the ones found as known.

ソースを見るGitHub で編集@johnmorrisdotca/hikidashi 1.1.0 のソースから作成しました。

この日本語は、まだ日本語を母語とする方の確認を受けていません。訂正を歓迎します。