Skip to content
Projects
Groups
Snippets
Help
Loading...
Help
Submit feedback
Contribute to GitLab
Sign in
Toggle navigation
C
coopwire-hypothesis
Project
Project
Details
Activity
Releases
Cycle Analytics
Repository
Repository
Files
Commits
Branches
Tags
Contributors
Graph
Compare
Charts
Issues
0
Issues
0
List
Board
Labels
Milestones
Merge Requests
0
Merge Requests
0
CI / CD
CI / CD
Pipelines
Jobs
Schedules
Charts
Wiki
Wiki
Snippets
Snippets
Members
Members
Collapse sidebar
Close sidebar
Activity
Graph
Charts
Create a new issue
Jobs
Commits
Issue Boards
Open sidebar
孙灵跃 Leon Sun
coopwire-hypothesis
Commits
fb56caf4
Commit
fb56caf4
authored
Dec 15, 2022
by
Robert Knight
Browse files
Options
Browse Files
Download
Email Patches
Plain Diff
Migrate ImageTextLayer and geometry utils to TS
parent
9a7de08c
Changes
2
Hide whitespace changes
Inline
Side-by-side
Showing
2 changed files
with
59 additions
and
90 deletions
+59
-90
image-text-layer.ts
src/annotator/integrations/image-text-layer.ts
+50
-54
geometry.ts
src/annotator/util/geometry.ts
+9
-36
No files found.
src/annotator/integrations/image-text-layer.
j
s
→
src/annotator/integrations/image-text-layer.
t
s
View file @
fb56caf4
import
debounce
from
'lodash.debounce'
;
import
debounce
from
'lodash.debounce'
;
import
type
{
DebouncedFunction
}
from
'lodash.debounce'
;
import
{
ListenerCollection
}
from
'../../shared/listener-collection'
;
import
{
ListenerCollection
}
from
'../../shared/listener-collection'
;
import
{
import
{
...
@@ -8,23 +9,24 @@ import {
...
@@ -8,23 +9,24 @@ import {
unionRects
,
unionRects
,
}
from
'../util/geometry'
;
}
from
'../util/geometry'
;
/**
type
WordBox
=
{
* @typedef WordBox
text
:
string
;
* @prop {string} text
* @prop {DOMRect} rect - Bounding rectangle of all glyphs in word
*/
/**
/** Bounding rect of all glyphs in word. */
* @typedef LineBox
rect
:
DOMRect
;
* @prop {WordBox[]} words
};
* @prop {DOMRect} rect - Bounding rectangle of all words in line
*/
/**
type
LineBox
=
{
* @typedef ColumnBox
words
:
WordBox
[];
* @prop {LineBox[]} lines
/** Bounding rect of all words in line. */
* @prop {DOMRect} rect - Bounding rectangle of all lines in column
rect
:
DOMRect
;
*/
};
type
ColumnBox
=
{
lines
:
LineBox
[];
/** Bounding rect of all lines in column. */
rect
:
DOMRect
;
};
/**
/**
* Group characters in a page into words, lines and columns.
* Group characters in a page into words, lines and columns.
...
@@ -36,16 +38,13 @@ import {
...
@@ -36,16 +38,13 @@ import {
* lines or columns that significantly intersect, as this can impair text
* lines or columns that significantly intersect, as this can impair text
* selection.
* selection.
*
*
* @param {DOMRect[]} charBoxes - Bounding rectangle associated with each character on the page
* @param charBoxes - Bounding rectangle associated with each character on the page
* @param {string} text - Text that corresponds to `charBoxes`
* @param text - Text that corresponds to `charBoxes`
* @return {ColumnBox[]}
*/
*/
function
analyzeLayout
(
charBoxes
,
text
)
{
function
analyzeLayout
(
charBoxes
:
DOMRect
[],
text
:
string
):
ColumnBox
[]
{
/** @type {WordBox[]} */
const
words
=
[]
as
WordBox
[];
const
words
=
[];
/** @type {WordBox} */
let
currentWord
=
{
text
:
''
,
rect
:
new
DOMRect
()
}
as
WordBox
;
let
currentWord
=
{
text
:
''
,
rect
:
new
DOMRect
()
};
// Group characters into words.
// Group characters into words.
const
addWord
=
()
=>
{
const
addWord
=
()
=>
{
...
@@ -54,7 +53,7 @@ function analyzeLayout(charBoxes, text) {
...
@@ -54,7 +53,7 @@ function analyzeLayout(charBoxes, text) {
currentWord
=
{
text
:
''
,
rect
:
new
DOMRect
()
};
currentWord
=
{
text
:
''
,
rect
:
new
DOMRect
()
};
}
}
};
};
for
(
le
t
[
i
,
rect
]
of
charBoxes
.
entries
())
{
for
(
cons
t
[
i
,
rect
]
of
charBoxes
.
entries
())
{
const
char
=
text
[
i
];
const
char
=
text
[
i
];
const
isSpace
=
/
\s
/
.
test
(
char
);
const
isSpace
=
/
\s
/
.
test
(
char
);
...
@@ -69,11 +68,9 @@ function analyzeLayout(charBoxes, text) {
...
@@ -69,11 +68,9 @@ function analyzeLayout(charBoxes, text) {
}
}
addWord
();
addWord
();
/** @type {LineBox[]} */
const
lines
=
[]
as
LineBox
[];
const
lines
=
[];
/** @type {LineBox} */
let
currentLine
=
{
words
:
[],
rect
:
new
DOMRect
()
}
as
LineBox
;
let
currentLine
=
{
words
:
[],
rect
:
new
DOMRect
()
};
// Group words into lines.
// Group words into lines.
const
addLine
=
()
=>
{
const
addLine
=
()
=>
{
...
@@ -82,7 +79,7 @@ function analyzeLayout(charBoxes, text) {
...
@@ -82,7 +79,7 @@ function analyzeLayout(charBoxes, text) {
currentLine
=
{
words
:
[],
rect
:
new
DOMRect
()
};
currentLine
=
{
words
:
[],
rect
:
new
DOMRect
()
};
}
}
};
};
for
(
le
t
word
of
words
)
{
for
(
cons
t
word
of
words
)
{
const
prevWord
=
currentLine
.
words
[
currentLine
.
words
.
length
-
1
];
const
prevWord
=
currentLine
.
words
[
currentLine
.
words
.
length
-
1
];
if
(
prevWord
)
{
if
(
prevWord
)
{
const
prevCenter
=
rectCenter
(
prevWord
.
rect
);
const
prevCenter
=
rectCenter
(
prevWord
.
rect
);
...
@@ -101,11 +98,9 @@ function analyzeLayout(charBoxes, text) {
...
@@ -101,11 +98,9 @@ function analyzeLayout(charBoxes, text) {
}
}
addLine
();
addLine
();
/** @type {ColumnBox[]} */
const
columns
=
[]
as
ColumnBox
[];
const
columns
=
[];
/** @type {ColumnBox} */
let
currentColumn
=
{
lines
:
[],
rect
:
new
DOMRect
()
}
as
ColumnBox
;
let
currentColumn
=
{
lines
:
[],
rect
:
new
DOMRect
()
};
// Group lines into columns.
// Group lines into columns.
const
addColumn
=
()
=>
{
const
addColumn
=
()
=>
{
...
@@ -114,7 +109,7 @@ function analyzeLayout(charBoxes, text) {
...
@@ -114,7 +109,7 @@ function analyzeLayout(charBoxes, text) {
currentColumn
=
{
lines
:
[],
rect
:
new
DOMRect
()
};
currentColumn
=
{
lines
:
[],
rect
:
new
DOMRect
()
};
}
}
};
};
for
(
le
t
line
of
lines
)
{
for
(
cons
t
line
of
lines
)
{
const
prevLine
=
currentColumn
.
lines
[
currentColumn
.
lines
.
length
-
1
];
const
prevLine
=
currentColumn
.
lines
[
currentColumn
.
lines
.
length
-
1
];
if
(
prevLine
)
{
if
(
prevLine
)
{
...
@@ -156,23 +151,29 @@ function analyzeLayout(charBoxes, text) {
...
@@ -156,23 +151,29 @@ function analyzeLayout(charBoxes, text) {
* viewer.
* viewer.
*/
*/
export
class
ImageTextLayer
{
export
class
ImageTextLayer
{
container
:
HTMLElement
;
private
_imageSizeObserver
?:
ResizeObserver
;
private
_listeners
:
ListenerCollection
;
private
_updateTextLayerSize
:
DebouncedFunction
<
[]
>
;
/**
/**
* Create a text layer which is displayed on top of `image`.
* Create a text layer which is displayed on top of `image`.
*
*
* @param
{Element}
image - Rendered image on which to overlay the text layer.
* @param image - Rendered image on which to overlay the text layer.
* The text layer will be inserted into the DOM as the next sibling of `image`.
* The text layer will be inserted into the DOM as the next sibling of `image`.
* @param
{DOMRect[]}
charBoxes - Bounding boxes for characters in the image.
* @param charBoxes - Bounding boxes for characters in the image.
* Coordinates should be in the range [0-1], where 0 is the top/left corner
* Coordinates should be in the range [0-1], where 0 is the top/left corner
* of the image and 1 is the bottom/right.
* of the image and 1 is the bottom/right.
* @param
{string}
text - Characters in the image corresponding to `charBoxes`
* @param text - Characters in the image corresponding to `charBoxes`
*/
*/
constructor
(
image
,
charBoxes
,
text
)
{
constructor
(
image
:
Element
,
charBoxes
:
DOMRect
[],
text
:
string
)
{
if
(
charBoxes
.
length
!==
text
.
length
)
{
if
(
charBoxes
.
length
!==
text
.
length
)
{
throw
new
Error
(
'Char boxes length does not match text length'
);
throw
new
Error
(
'Char boxes length does not match text length'
);
}
}
// Create container for text layer and position it above the image.
// Create container for text layer and position it above the image.
const
containerParent
=
/** @type {HTMLElement} */
(
image
.
parentNode
)
;
const
containerParent
=
image
.
parentNode
as
HTMLElement
;
const
container
=
document
.
createElement
(
'hypothesis-text-layer'
);
const
container
=
document
.
createElement
(
'hypothesis-text-layer'
);
containerParent
.
insertBefore
(
container
,
image
.
nextSibling
);
containerParent
.
insertBefore
(
container
,
image
.
nextSibling
);
...
@@ -201,20 +202,15 @@ export class ImageTextLayer {
...
@@ -201,20 +202,15 @@ export class ImageTextLayer {
container
.
style
.
fontSize
=
fontSize
+
'px'
;
container
.
style
.
fontSize
=
fontSize
+
'px'
;
container
.
style
.
fontFamily
=
fontFamily
;
container
.
style
.
fontFamily
=
fontFamily
;
const
canvas
=
document
.
createElement
(
'canvas'
);
const
canvas
=
document
.
createElement
(
'canvas'
);
const
context
=
/** @type {CanvasRenderingContext2D} */
(
const
context
=
canvas
.
getContext
(
'2d'
)
as
CanvasRenderingContext2D
;
canvas
.
getContext
(
'2d'
)
);
context
.
font
=
`
${
fontSize
}
px
${
fontFamily
}
`
;
context
.
font
=
`
${
fontSize
}
px
${
fontFamily
}
`
;
/**
/** Generate a CSS value that scales with the `--x-scale` or `--y-scale` CSS variables. */
* Generate a CSS value that scales with the `--x-scale` or `--y-scale` CSS variables.
const
scaledValue
=
(
*
dimension
:
'x'
|
'y'
,
* @param {'x'|'y'} dimension
value
:
number
,
* @param {number} value
unit
=
'px'
as
string
* @param {string} unit
)
=>
`calc(var(--
${
dimension
}
-scale) *
${
value
}${
unit
}
)`
;
*/
const
scaledValue
=
(
dimension
,
value
,
unit
=
'px'
)
=>
`calc(var(--
${
dimension
}
-scale) *
${
value
}${
unit
}
)`
;
// Group characters into words, lines and columns. Then use the result to
// Group characters into words, lines and columns. Then use the result to
// create a hierarchical DOM structure in the text layer:
// create a hierarchical DOM structure in the text layer:
...
@@ -227,7 +223,7 @@ export class ImageTextLayer {
...
@@ -227,7 +223,7 @@ export class ImageTextLayer {
// in-between lines or words.
// in-between lines or words.
const
columns
=
analyzeLayout
(
charBoxes
,
text
);
const
columns
=
analyzeLayout
(
charBoxes
,
text
);
for
(
le
t
column
of
columns
)
{
for
(
cons
t
column
of
columns
)
{
const
columnEl
=
document
.
createElement
(
'hypothesis-text-column'
);
const
columnEl
=
document
.
createElement
(
'hypothesis-text-column'
);
columnEl
.
style
.
display
=
'block'
;
columnEl
.
style
.
display
=
'block'
;
columnEl
.
style
.
position
=
'absolute'
;
columnEl
.
style
.
position
=
'absolute'
;
...
@@ -235,7 +231,7 @@ export class ImageTextLayer {
...
@@ -235,7 +231,7 @@ export class ImageTextLayer {
columnEl
.
style
.
top
=
scaledValue
(
'y'
,
column
.
rect
.
top
);
columnEl
.
style
.
top
=
scaledValue
(
'y'
,
column
.
rect
.
top
);
let
prevLine
=
null
;
let
prevLine
=
null
;
for
(
le
t
line
of
column
.
lines
)
{
for
(
cons
t
line
of
column
.
lines
)
{
const
lineEl
=
document
.
createElement
(
'hypothesis-text-line'
);
const
lineEl
=
document
.
createElement
(
'hypothesis-text-line'
);
lineEl
.
style
.
display
=
'block'
;
lineEl
.
style
.
display
=
'block'
;
lineEl
.
style
.
marginLeft
=
scaledValue
(
lineEl
.
style
.
marginLeft
=
scaledValue
(
...
@@ -256,7 +252,7 @@ export class ImageTextLayer {
...
@@ -256,7 +252,7 @@ export class ImageTextLayer {
lineEl
.
style
.
whiteSpace
=
'nowrap'
;
lineEl
.
style
.
whiteSpace
=
'nowrap'
;
let
prevWord
=
null
;
let
prevWord
=
null
;
for
(
le
t
word
of
line
.
words
)
{
for
(
cons
t
word
of
line
.
words
)
{
const
wordEl
=
document
.
createElement
(
'hypothesis-text-word'
);
const
wordEl
=
document
.
createElement
(
'hypothesis-text-word'
);
wordEl
.
style
.
display
=
'inline-block'
;
wordEl
.
style
.
display
=
'inline-block'
;
wordEl
.
style
.
transformOrigin
=
'top left'
;
wordEl
.
style
.
transformOrigin
=
'top left'
;
...
...
src/annotator/util/geometry.
j
s
→
src/annotator/util/geometry.
t
s
View file @
fb56caf4
/**
/**
* Return the intersection of two rects.
* Return the intersection of two rects.
*
* @param {DOMRect} rectA
* @param {DOMRect} rectB
*/
*/
export
function
intersectRects
(
rectA
,
rectB
)
{
export
function
intersectRects
(
rectA
:
DOMRect
,
rectB
:
DOMRect
)
{
const
left
=
Math
.
max
(
rectA
.
left
,
rectB
.
left
);
const
left
=
Math
.
max
(
rectA
.
left
,
rectB
.
left
);
const
right
=
Math
.
min
(
rectA
.
right
,
rectB
.
right
);
const
right
=
Math
.
min
(
rectA
.
right
,
rectB
.
right
);
const
top
=
Math
.
max
(
rectA
.
top
,
rectB
.
top
);
const
top
=
Math
.
max
(
rectA
.
top
,
rectB
.
top
);
...
@@ -18,10 +15,8 @@ export function intersectRects(rectA, rectB) {
...
@@ -18,10 +15,8 @@ export function intersectRects(rectA, rectB) {
* An empty rect is defined as one with zero or negative width/height, eg.
* An empty rect is defined as one with zero or negative width/height, eg.
* as returned by `new DOMRect()` or `Element.getBoundingClientRect()` for a
* as returned by `new DOMRect()` or `Element.getBoundingClientRect()` for a
* hidden element.
* hidden element.
*
* @param {DOMRect} rect
*/
*/
export
function
rectIsEmpty
(
rect
)
{
export
function
rectIsEmpty
(
rect
:
DOMRect
)
{
return
rect
.
width
<=
0
||
rect
.
height
<=
0
;
return
rect
.
width
<=
0
||
rect
.
height
<=
0
;
}
}
...
@@ -35,13 +30,8 @@ export function rectIsEmpty(rect) {
...
@@ -35,13 +30,8 @@ export function rectIsEmpty(rect) {
* c------d
* c------d
*
*
* The inputs must be normalized such that b >= a and d >= c.
* The inputs must be normalized such that b >= a and d >= c.
*
* @param {number} a
* @param {number} b
* @param {number} c
* @param {number} d
*/
*/
function
linesOverlap
(
a
,
b
,
c
,
d
)
{
function
linesOverlap
(
a
:
number
,
b
:
number
,
c
:
number
,
d
:
number
)
{
const
maxStart
=
Math
.
max
(
a
,
c
);
const
maxStart
=
Math
.
max
(
a
,
c
);
const
minEnd
=
Math
.
min
(
b
,
d
);
const
minEnd
=
Math
.
min
(
b
,
d
);
return
maxStart
<
minEnd
;
return
maxStart
<
minEnd
;
...
@@ -49,11 +39,8 @@ function linesOverlap(a, b, c, d) {
...
@@ -49,11 +39,8 @@ function linesOverlap(a, b, c, d) {
/**
/**
* Return true if the intersection of `rectB` and `rectA` is non-empty.
* Return true if the intersection of `rectB` and `rectA` is non-empty.
*
* @param {DOMRect} rectA
* @param {DOMRect} rectB
*/
*/
export
function
rectIntersects
(
rectA
,
rectB
)
{
export
function
rectIntersects
(
rectA
:
DOMRect
,
rectB
:
DOMRect
)
{
if
(
rectIsEmpty
(
rectA
)
||
rectIsEmpty
(
rectB
))
{
if
(
rectIsEmpty
(
rectA
)
||
rectIsEmpty
(
rectB
))
{
return
false
;
return
false
;
}
}
...
@@ -66,11 +53,8 @@ export function rectIntersects(rectA, rectB) {
...
@@ -66,11 +53,8 @@ export function rectIntersects(rectA, rectB) {
/**
/**
* Return true if `rectB` is fully contained within `rectA`
* Return true if `rectB` is fully contained within `rectA`
*
* @param {DOMRect} rectA
* @param {DOMRect} rectB
*/
*/
export
function
rectContains
(
rectA
,
rectB
)
{
export
function
rectContains
(
rectA
:
DOMRect
,
rectB
:
DOMRect
)
{
if
(
rectIsEmpty
(
rectA
)
||
rectIsEmpty
(
rectB
))
{
if
(
rectIsEmpty
(
rectA
)
||
rectIsEmpty
(
rectB
))
{
return
false
;
return
false
;
}
}
...
@@ -85,21 +69,15 @@ export function rectContains(rectA, rectB) {
...
@@ -85,21 +69,15 @@ export function rectContains(rectA, rectB) {
/**
/**
* Return true if two rects overlap vertically.
* Return true if two rects overlap vertically.
*
* @param {DOMRect} a
* @param {DOMRect} b
*/
*/
export
function
rectsOverlapVertically
(
a
,
b
)
{
export
function
rectsOverlapVertically
(
a
:
DOMRect
,
b
:
DOMRect
)
{
return
linesOverlap
(
a
.
top
,
a
.
bottom
,
b
.
top
,
b
.
bottom
);
return
linesOverlap
(
a
.
top
,
a
.
bottom
,
b
.
top
,
b
.
bottom
);
}
}
/**
/**
* Return true if two rects overlap horizontally.
* Return true if two rects overlap horizontally.
*
* @param {DOMRect} a
* @param {DOMRect} b
*/
*/
export
function
rectsOverlapHorizontally
(
a
,
b
)
{
export
function
rectsOverlapHorizontally
(
a
:
DOMRect
,
b
:
DOMRect
)
{
return
linesOverlap
(
a
.
left
,
a
.
right
,
b
.
left
,
b
.
right
);
return
linesOverlap
(
a
.
left
,
a
.
right
,
b
.
left
,
b
.
right
);
}
}
...
@@ -109,11 +87,8 @@ export function rectsOverlapHorizontally(a, b) {
...
@@ -109,11 +87,8 @@ export function rectsOverlapHorizontally(a, b) {
* The union of an empty rect (see {@link rectIsEmpty}) with a non-empty rect is
* The union of an empty rect (see {@link rectIsEmpty}) with a non-empty rect is
* defined to be the non-empty rect. The union of two empty rects is an empty
* defined to be the non-empty rect. The union of two empty rects is an empty
* rect.
* rect.
*
* @param {DOMRect} a
* @param {DOMRect} b
*/
*/
export
function
unionRects
(
a
,
b
)
{
export
function
unionRects
(
a
:
DOMRect
,
b
:
DOMRect
)
{
if
(
rectIsEmpty
(
a
))
{
if
(
rectIsEmpty
(
a
))
{
return
b
;
return
b
;
}
else
if
(
rectIsEmpty
(
b
))
{
}
else
if
(
rectIsEmpty
(
b
))
{
...
@@ -130,10 +105,8 @@ export function unionRects(a, b) {
...
@@ -130,10 +105,8 @@ export function unionRects(a, b) {
/**
/**
* Return the point at the center of a rect.
* Return the point at the center of a rect.
*
* @param {DOMRect} rect
*/
*/
export
function
rectCenter
(
rect
)
{
export
function
rectCenter
(
rect
:
DOMRect
)
{
return
new
DOMPoint
(
return
new
DOMPoint
(
(
rect
.
left
+
rect
.
right
)
/
2
,
(
rect
.
left
+
rect
.
right
)
/
2
,
(
rect
.
top
+
rect
.
bottom
)
/
2
(
rect
.
top
+
rect
.
bottom
)
/
2
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment