Repository navigation
Expand file tree
/
Copy pathstart.html
More file actions
365 lines (337 loc) · 14.6 KB
/
Copy pathstart.html
File metadata and controls
365 lines (337 loc) · 14.6 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8">
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<meta name="viewport" content="width=device-width, initial-scale=1">
<meta name="description" content="Search the web together">
<meta name="author" content="searsia.org">
<link rel="icon" href="images/searsia.png">
<title>Searsia - Start</title>
<link href="css/bootstrap.min.css" rel="stylesheet">
<link href="css/searsia-doc.css" rel="stylesheet">
</head>
<body class="searsia-doc">
<div class="container">
<!-- Static navbar -->
<nav class="navbar navbar-default">
<div class="container-fluid">
<div class="navbar-header">
<button type="button" class="navbar-toggle collapsed" data-toggle="collapse" data-target="#navbar" aria-expanded="false" aria-controls="navbar">
<span class="sr-only">Toggle navigation</span>
<span class="icon-bar"></span>
<span class="icon-bar"></span>
<span class="icon-bar"></span>
</button>
<a class="navbar-brand" href="index.html">Searsia</a>
</div>
<div id="navbar" class="navbar-collapse collapse">
<ul class="nav navbar-nav">
<li><a href="about.html">About</a></li>
<li class="active"><a href="">Start</a></li>
<li><a href="protocol.html">Protocol</a></li>
<li><a href="people.html">People</a></li>
<li><a href="/blog/">Blog</a></li>
</ul>
<div class="col-sm-3 pull-right">
<form id="searsia-form" class="navbar-form" action="search.html" method="get">
<div class="input-group input-group-sm">
<input type="text" class="form-control" placeholder="search" name="q" id="query">
<div class="input-group-btn">
<button class="btn btn-default" type="submit"><i class="glyphicon glyphicon-search"></i></button>
</div>
</div>
</form>
</div>
</div><!--/.nav-collapse -->
</div><!--/.container-fluid -->
</nav>
<div class="row">
<div class="col-md-8">
<h1>Start <img src="images/searsia.png" alt=""></h1>
<p>Searsia comes with a client and a server.</p>
<h2 id="client">The client</h2>
<p>The Searsia Web client can be downloaded as
<a href="https://github.com/searsia/searsiaclient/releases/download/v1.0.2/searsiaclient.zip" rel="nofollow"><strong>searsiaclient.zip</strong></a>
and unzipped on your local machine or web server. To use the web client:
</p>
<ol>
<li>Set the variable <code>API_TEMPLATE</code> in the second line of the file <code>js/searsia.js</code>;</li>
<li>Open <code>index.html</code> in a web browser;</li>
<li>Congratulations! You now run your own web application for federated search.</li>
</ol>
<p> The API template is a url with a placeholder for the query and possibly
other parameters. Examples of API templates of on-line Searsia servers are
the University of Twente's site search engine, and Dr. Sheet Music:
</p>
<p>
<code>https://search.utwente.nl/searsia/index.json?q={searchTerms}&page={startPage?}</code>
</p>
<p> or </p>
<p>
<code>https://drsheetmusic.com/searsia/index.json?q={searchTerms}&page={startPage?}</code>
</p>
<p>
If you run Searsia Server on your local machine (see next section), you can
connect the client to your own server by setting the API template to something
like:
</p>
<p>
<code>http://localhost:16842/searsia/index?q={searchTerms?}&page={startPage?}</code>
</p>
<h2 id="server">The server</h2>
<p>Download the Java server
<a href="https://github.com/searsia/searsiaserver/releases/download/v1.0.2/searsiaserver.jar" rel="nofollow"><strong>searsiaserver.jar</strong></a>
and use the following command to start the server:
</p>
<p>
<code class="block"> java -jar searsiaserver.jar </code>
</p>
<p>
The server requires Java 8 or higher. Just like the web client, the java
server needs another Searsia server's API template to connect to. We call
the other server the <em>mother</em>, because your server will learn
what it needs to know from the other server.
Your server will display the following message:
</p>
<p>
<code class="result">
Please provide mother's api template (use '-m').
</code>
</p>
<p>
Additionally, the server displays all <a href="#serveroptions">server options</a>.
Use the option <code>-m</code> to provide the API template, for instance
one of the example templates under <a href="#clientoptions">Client Options</a>
above.
</p>
<p>
Your Searsia server copies the Searsia search engine definitions of the
server it connects to (specified by the API template).
Your server will have its own API template, which it will display at
start up. For instance, connect to Dr. Sheet Music as follows:
</p>
<p>
<code class="block"> java -jar searsiaserver.jar -m 'https://drsheetmusic.com/searsia/index.json?q={searchTerms}&page={startPage?}'</code>
</p>
<p>
<code class="result">
Searsia server v1.0.2<br>
Starting: Dr. Sheet Music (index)<br>
API template: http://localhost:16842/searsia/index?q={searchTerms}&page={startPage?}<br>
Use Ctrl+c to stop.
</code>
</p>
<p>
Use the reported API template in your <a href="#client">client</a> as
explained above to connect your client to your server.
</p>
<h2 id="federation">Federation: One server, many search engines</h2>
<p>
A Searsia server provides access to many search engines. Together, the search
engines form a federation. Like a federation of countries that form a country
together, Searsia manages a federation of search engines that form a search
engine together. Each search engine in the federation has a unique identifier.
The identifier of the search engine in the examples above is '<code>index</code>'.
Suppose there is another search engine in the federation that is called
'<code>didyoumean</code>', then we can access it by replacing '<code>index</code>'
by '<code>didyoumean</code>' as follows:
</p>
<p>
<code> <a href="https://search.utwente.nl/searsia/didyoumean">https://search.utwente.nl/searsia/didyoumean</a> </code>
</p>
<p> If you look at the URL in your browser, you will see the JSON search engine
definition below:
</p>
<pre>
{
"resource": {
"apitemplate": "https://search.utwente.nl/searsia/didyoumean.php?q={searchTerms}",
"favicon": "https://search.utwente.nl/ut-icons/ut.png",
"id": "didyoumean",
"mimetype": "application/searsia+json",
"name": "Did you mean:",
"testquery": "test"
},
"searsia": "v1.0.2"
}
</pre>
<p>
Searsia servers copy and share these search engine definition files.
We might now start a local Searsia server that serves search results
from '<code>didyoumean</code>' as follows:
</p>
<p>
<code class="block"> java -jar searsiaserver.jar -m 'https://search.utwente.nl/searsia/didyoumean?q={searchTerms}'</code>
</p>
<p>
Searsia retrieves the definition file and runs a local copy that
provides results from '<code>didyoumean</code>'.
In fact, we might also download the JSON definition file to our machine
and then start Searsia Server from there with the same effect:
</p>
<p>
<code class="block">
wget https://search.utwente.nl/searsia/didyoumean.json<br>
java -jar searsiaserver.jar -m didyoumean.json
</code>
</p>
<h2 id="testing">Testing a search engine</h2>
<p>
Searsia Server tests a search engine using the <code>-t</code> option as follows:
</p>
<p>
<code class="block">
java -jar searsiaserver.jar -m didyoumean.json -t json
</code>
</p>
<p>
This will output the <code>json</code> search results for the <code>"testquery"</code> that
is specified in <code>didyoumean.json</code>, or an error if there is a
problem with the definition file or the <code>didyoumean</code> search engine.
</p>
<p>
Testing search engines will be needed frequently when setting up a new Searsia
engine, or when maintaining an existing Searsia engine. More about adding
engines to the federation can be found on the <a href="protocol.html">Protocol</a>
page.
<h2 id="serveroptions">Server options</h2>
<p>
Searsia server supports several command line options, shown in the following
table. For convenience, each option has a short-hand consisting of one hyphen
and the first letter of the option, for instance <code>-h</code> for
<code>--help</code>.
</p>
<table class="table table-bordered">
<tr><th>Option</th><th>Explanation</th></tr>
<tr id="option-cache"><td>--cache <arg></td>
<td> Set cache size (integer: number of result pages). The default is 500
pages.</td>
</tr>
<tr id="option-dontshare"><td>--dontshare</td>
<td> Do not share resource definitions. This will make it impossible for
other servers to meaningfully connect to your server.</td>
</tr>
<tr id="option-export"><td>--export</td>
<td>Export index to stdout and exit.</td>
</tr>
<tr id="option-help"><td> --help</td>
<td> Show help.</td>
</tr>
<tr id="option-interval"><td style="white-space:nowrap">--interval <arg></td>
<td>Takes as argument an integer, which is the poll interval (in seconds). The
server sends a random query each interval to a search engine.
The default value is <code>120</code> seconds.
If your server contains 30 resources (search engines), a polling interval of 120
seconds will poll each resource on average once per 120 * 30 = 3600 seconds,
so once each 60 minutes, about 24 queries a day. In scientific literature on
distributed information retrieval, polling is usually called <em>query-based sampling</em>.
An interval of <code>0</code> disables polling.</td>
</tr>
<tr id="option-log"><td>--log <arg></td><td>Takes as argument an integer, which is the
type of log messages produced by the server (to the <a href="#option-path">index directory</a>).
Supported levels are: 0 = no logging; 1 = only errors; 2 = errors and warnings; 3 = errors, warnings,
and information, 4 = all of level 3 plus debug information. The default level is 2.</td>
</tr>
<tr id="option-mother"><td>--mother <arg></td>
<td>REQUIRED. Sets the API template of the mother. See above for example values.
</td>
</tr>
<tr id="option-health"><td>--nohealth</td>
<td>Do not share health report.</td>
</tr>
<tr id="option-path"><td>--path <arg></td>
<td>Set the path on the file system where the index is stored. The default depends on your
operating system. Typically, the index ends up in your home directory under:
<code>.local/share/searsia</code> if your are on a Linux-based system,
under <code>Library/Application Support/Searsia</code> if you are on Apple,
and under <code>Application Data/Searsia</code> on many Windows versions.
</td>
</tr>
<tr id="option-quiet"><td>--quiet</td>
<td>No output to console.</td>
</tr>
<tr id="option-test"><td>--test <arg></td>
<td> Print test output and exit (argument: 'json', 'xml', 'response', or 'all').</td>
</tr>
<tr id="option-url"><td>--url <arg></td>
<td>Set the url of the web service endpoint. The default is '<code>http://localhost:16842/searsia/</code>'</td>
</tr>
</table>
<h2 id="troubleshooting">Server trouble shooting</h2>
<p>
The server might return the following error messages:
</p>
<ul>
<li id="error-create-directory"><code>Setup failed: index_75cbc797a03dfe0dd08780e6c68e7bbc </code><br>
The server cannot create the index. Check if the server has write premissions in the reported directory.
Use the <a href="#option-path">--path</a> option to change the location of the index.
</li>
<li id="error-timed-out"><code>Setup failed: Lock obtain timed out: NativeFSLock</code><br>
There is already an instance of the Searsia server running on the same index.
Kill the server and try again.
</li>
<li id="error-start-server"><code>Server failed: Failed to start Grizzly HTTP server: Address already in use</code><br>
There is already a running instance of the Searsia server on the same url.
Kill the server, or run the server on a different url or port number with the <a href="#option-url">--url</a> option.
</li>
<li id="error-server-address"><code>Server failed: Failed to start Grizzly HTTP server: Unresolved address</code>, or<br>
<code>Server failed: Failed to start Grizzly HTTP server: Permission denied</code><br>
The provided url is invalid, or the url does not belong to the machine that runs the server, or the port is blocked or taken by another application.
Change the url with the <a href="#option-url">--url</a> option.
</li>
<li id="error-unknownhost-exception"><code>Error: Connection failed: java.net.UnknownHostException</code>, or<br>
<code>Error: Connection failed: java.io.FileNotFoundException</code><br>
Connection to the mother failed. Check your internet connection. Check the mother engine and mother template.
Change the mother template with the <a href="#option-mother">--mother</a> option.
</li>
<li id="error-json-exception"><code>Error: Connection failed: org.json.JSONException</code><br>
The mother is not a Searsia engine. Check the mother engine and mother template.
Change the mother with the <a href="#option-mother">--mother</a> option.
</li>
</ul>
</div>
<div class="col-sm-4">
<div class="panel panel-default" style="margin-top:2em">
<div class="panel-heading">
<h3 class="panel-title">Download Searsia </h3>
</div>
<div class="panel-body">
<p>
<a href="http://github.com/searsia/searsiaclient/releases/download/v1.0.2/searsiaclient.zip" rel="nofollow">
<img src="images/searsia.png" alt="" width="20"/>
<strong>searsiaclient.zip</strong></a>
<span style="white-space:nowrap">(213 KB)</span>
</p>
<p>
<a href="http://github.com/searsia/searsiaserver/releases/download/v1.0.2/searsiaserver.jar" rel="nofollow">
<img src="images/searsia.png" alt="" width="20"/>
<strong>searsiaserver.jar</strong></a>
<span style="white-space:nowrap">(11.2 MB)</span>
</p>
</div>
</div>
</div>
<div class="col-sm-4">
<div class="panel panel-default" style="margin-top:2em">
<div class="panel-heading">
<h3 class="panel-title">Thanks, Github!</h3>
</div>
<div class="panel-body">
<a target="_blank" href="https://github.com/searsia">
<img src="https://github.com/fluidicon.png" alt="" width="44"/>
<strong>hosted by Github</strong>
</a>
</div>
</div>
</div>
</div> <!-- row -->
<footer>
<hr>
<p><strong>Next:</strong> <a href="protocol.html">The Searsia protocol</a>.</p>
</footer>
</div> <!-- /container -->
<script src="js/jquery.min.js"></script>
<script src="js/bootstrap.min.js"></script>
</body>
</html>