java.lang.Object
- org.openstreetmap.josm.io.OsmConnection
- - org.openstreetmap.josm.io.OsmServerReader
  - - org.openstreetmap.josm.io.MultiFetchServerObjectReader

Direct Known Subclasses:: MultiFetchOverpassObjectReader

public class MultiFetchServerObjectReader
extends OsmServerReader

Retrieves a set of OsmPrimitives from an OSM server using the so called Multi Fetch API.

Usage:

    MultiFetchServerObjectReader reader = MultiFetchServerObjectReader()
         .append(new Node(72343));
    reader.parseOsm();
    if (!reader.getMissingPrimitives().isEmpty()) {
        Logging.info("There are missing primitives: " + reader.getMissingPrimitives());
    }
    if (!reader.getSkippedWays().isEmpty()) {
       Logging.info("There are skipped ways: " + reader.getMissingPrimitives());
    }

Nested Class Summary

Nested Classes
Modifier and Type	Class	Description
`protected class`	`MultiFetchServerObjectReader.Fetcher`	The class that actually download data from OSM API.
`protected static class`	`MultiFetchServerObjectReader.FetchResult`	The class holding the results given by `MultiFetchServerObjectReader.Fetcher`.

Nested classes/interfaces inherited from class org.openstreetmap.josm.io.OsmServerReader
OsmServerReader.DomParser<R>

Nested classes/interfaces inherited from class org.openstreetmap.josm.io.OsmConnection
OsmConnection.OAuthAccessTokenFetcher

Field Summary

Fields
Modifier and Type	Field	Description
`private java.util.concurrent.ExecutorService`	`exec`
`private static int`	`MAX_IDS_PER_REQUEST`	the max.
`private java.util.Set<PrimitiveId>`	`missingPrimitives`
`private java.util.Set<java.lang.Long>`	`nodes`
`private DataSet`	`outputDataSet`
`protected java.util.Map<OsmPrimitiveType,java.util.Set<java.lang.Long>>`	`primitivesMap`
`private boolean`	`recurseDownAppended`
`protected boolean`	`recurseDownRelations`
`private java.util.Set<java.lang.Long>`	`relations`
`private java.util.Set<java.lang.Long>`	`ways`

Fields inherited from class org.openstreetmap.josm.io.OsmServerReader
contentType, gpxParsedProperly

Fields inherited from class org.openstreetmap.josm.io.OsmConnection
activeConnection, cancel, fetcher, oAuth20Parameters

Constructor Summary

Constructors
Modifier Constructor Description

protected MultiFetchServerObjectReader()
Constructs a MultiFetchServerObjectReader.

Method Summary

All Methods Static Methods Instance Methods Concrete Methods
Modifier and Type	Method	Description
`MultiFetchServerObjectReader`	`append(java.util.Collection<? extends OsmPrimitive> primitives)`	appends a list of `OsmPrimitive` to the list of ids which will be fetched from the server.
`MultiFetchServerObjectReader`	`append(DataSet ds, long id, OsmPrimitiveType type)`	appends a `OsmPrimitive` id to the list of ids which will be fetched from the server.
`MultiFetchServerObjectReader`	`append(OsmPrimitive primitive)`	appends an `OsmPrimitive` to the list of ids which will be fetched from the server.
`void`	`append(PrimitiveId id)`	Remembers an `OsmPrimitive`'s id.
`MultiFetchServerObjectReader`	`appendNode(Node node)`	appends a `Node` id to the list of ids which will be fetched from the server.
`protected MultiFetchServerObjectReader`	`appendRelation(Relation relation)`	appends a `Relation` id to the list of ids which will be fetched from the server.
`MultiFetchServerObjectReader`	`appendWay(Way way)`	appends a `Way` id and the list of ids of nodes the way refers to the list of ids which will be fetched from the server.
`protected java.lang.String`	`buildRequestString(OsmPrimitiveType type, java.util.Set<java.lang.Long> idPackage)`	builds the Multi Get request string for a set of ids and a given `OsmPrimitiveType`.
`void`	`cancel()`	Cancels the connection.
`private void`	`checkMissing(DataSet ds, ProgressMonitor progressMonitor)`	Workaround for difference in Overpass API.
`static MultiFetchServerObjectReader`	`create()`	Creates a new instance of `MultiFetchServerObjectReader` or `MultiFetchOverpassObjectReader` depending on the `preference`.
`static MultiFetchServerObjectReader`	`create(boolean fromMirror)`	Creates a new instance of `MultiFetchServerObjectReader` or `MultiFetchOverpassObjectReader` depending on the `fromMirror` parameter.
`private void`	`downloadRelations(ProgressMonitor progressMonitor)`	Finds best way to download a set of relations.
`protected java.util.Set<java.lang.Long>`	`extractIdPackage(java.util.Set<java.lang.Long> ids)`	extracts a subset of max `MAX_IDS_PER_REQUEST` ids from `ids` and replies the subset.
`protected void`	`fetchPrimitives(java.util.Set<java.lang.Long> ids, OsmPrimitiveType type, ProgressMonitor progressMonitor)`	fetches a set of ids of a given `OsmPrimitiveType` from the server
`java.util.Set<PrimitiveId>`	`getMissingPrimitives()`	replies the set of ids of all primitives for which a fetch request to the server was submitted but which are not available from the server (the server replied a return code of 404)
`protected void`	`merge(DataSet from)`	merges the dataset `from` to `outputDataSet`.
`DataSet`	`parseOsm(ProgressMonitor progressMonitor)`	invokes one or more Multi Gets to fetch the `OsmPrimitive`s and replies the dataset of retrieved primitives.
`protected void`	`rememberNodesOfIncompleteWaysToLoad(DataSet from)`
`MultiFetchServerObjectReader`	`setRecurseDownAppended(boolean recurseAppended)`	Determine how appended objects are treated.
`MultiFetchServerObjectReader`	`setRecurseDownRelations(boolean recurseDownRelations)`	Should downloaded relations be complete?

Methods inherited from class org.openstreetmap.josm.io.OsmServerReader
adaptRequest, fetchData, getAttribute, getBaseUrl, getInputStream, getInputStream, getInputStreamRaw, getInputStreamRaw, getInputStreamRaw, getInputStreamRaw, isDoAuthenticate, isGpxParsedProperly, parseNotes, parseOsm, parseOsmChange, parseOsmChange, parseRawGps, parseRawGps, parseRawNotes, parseRawNotes, setDoAuthenticate

Methods inherited from class org.openstreetmap.josm.io.OsmConnection
addAuth, addBasicAuthorizationHeader, addOAuth20AuthorizationHeader, isCanceled, retrieveBasicAuthorizationLogin, setOAuthAccessTokenFetcher

Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

- Field Detail
  - MAX_IDS_PER_REQUEST
```
private static final int MAX_IDS_PER_REQUEST
```
    the max. number of primitives retrieved in one step. Assuming IDs with 10 digits, this leads to a max. request URL of ~ 1900 Bytes ((10 digits + 1 Separator) * 170), which should be safe according to the WWW FAQ.
    
    See Also:
    
    Constant Field Values
  - nodes
```
private final java.util.Set<java.lang.Long> nodes
```
  - ways
```
private final java.util.Set<java.lang.Long> ways
```
  - relations
```
private final java.util.Set<java.lang.Long> relations
```
  - missingPrimitives
```
private final java.util.Set<PrimitiveId> missingPrimitives
```
  - outputDataSet
```
private final DataSet outputDataSet
```
  - primitivesMap
```
protected final java.util.Map<OsmPrimitiveType,java.util.Set<java.lang.Long>> primitivesMap
```
  - recurseDownRelations
```
protected boolean recurseDownRelations
```
  - recurseDownAppended
```
private boolean recurseDownAppended
```
  - exec
```
private java.util.concurrent.ExecutorService exec
```
- Constructor Detail
  - MultiFetchServerObjectReader
```
protected MultiFetchServerObjectReader()
```
    Constructs a MultiFetchServerObjectReader.
- Method Detail
  - create
```
public static MultiFetchServerObjectReader create()
```
    Creates a new instance of MultiFetchServerObjectReader or MultiFetchOverpassObjectReader depending on the preference.
    
    Returns:
    
    a new instance
    
    Since:
    
    9241
  - create
```
public static MultiFetchServerObjectReader create(boolean fromMirror)
```
    Creates a new instance of MultiFetchServerObjectReader or MultiFetchOverpassObjectReader depending on the fromMirror parameter.
    
    Parameters:
    
    fromMirror - false for MultiFetchServerObjectReader, true for MultiFetchOverpassObjectReader
    
    Returns:
    
    a new instance
    
    Since:
    
    15520 (changed visibility)
  - append
```
public void append(PrimitiveId id)
```
    Remembers an OsmPrimitive's id. The id will later be fetched as part of a Multi Get request.
    Ignore the id if it represents a new primitives.
    
    Parameters:
    
    id - the id
  - append
```
public MultiFetchServerObjectReader append(DataSet ds,
                                           long id,
                                           OsmPrimitiveType type)
```
    appends a OsmPrimitive id to the list of ids which will be fetched from the server.
    
    Parameters:
    
    ds - the DataSet to which the primitive belongs
    
    id - the primitive id
    
    type - The primitive type. Must be one of NODE, WAY, RELATION
    
    Returns:
    
    this
  - appendNode
```
public MultiFetchServerObjectReader appendNode(Node node)
```
    appends a Node id to the list of ids which will be fetched from the server.
    
    Parameters:
    
    node - the node (ignored, if null)
    
    Returns:
    
    this
  - appendWay
```
public MultiFetchServerObjectReader appendWay(Way way)
```
    appends a Way id and the list of ids of nodes the way refers to the list of ids which will be fetched from the server.
    
    Parameters:
    
    way - the way (ignored, if null)
    
    Returns:
    
    this
  - appendRelation
```
protected MultiFetchServerObjectReader appendRelation(Relation relation)
```
    appends a Relation id to the list of ids which will be fetched from the server.
    
    Parameters:
    
    relation - the relation (ignored, if null)
    
    Returns:
    
    this
  - append
```
public MultiFetchServerObjectReader append(OsmPrimitive primitive)
```
    appends an OsmPrimitive to the list of ids which will be fetched from the server.
    
    Parameters:
    
    primitive - the primitive
    
    Returns:
    
    this
  - append
```
public MultiFetchServerObjectReader append(java.util.Collection<? extends OsmPrimitive> primitives)
```
    appends a list of OsmPrimitive to the list of ids which will be fetched from the server.
    
    Parameters:
    
    primitives - the list of primitives (ignored, if null)
    
    Returns:
    
    this
    
    See Also:
    
    append(OsmPrimitive)
  - extractIdPackage
```
protected java.util.Set<java.lang.Long> extractIdPackage(java.util.Set<java.lang.Long> ids)
```
    extracts a subset of max MAX_IDS_PER_REQUEST ids from ids and replies the subset. The extracted subset is removed from ids.
    
    Parameters:
    
    ids - a set of ids
    
    Returns:
    
    the subset of ids
  - buildRequestString
```
protected java.lang.String buildRequestString(OsmPrimitiveType type,
                                              java.util.Set<java.lang.Long> idPackage)
```
    builds the Multi Get request string for a set of ids and a given OsmPrimitiveType.
    
    Parameters:
    
    type - The primitive type. Must be one of NODE, WAY, RELATION
    
    idPackage - the package of ids
    
    Returns:
    
    the request string
  - rememberNodesOfIncompleteWaysToLoad
```
protected void rememberNodesOfIncompleteWaysToLoad(DataSet from)
```
  - merge
```
protected void merge(DataSet from)
```
    merges the dataset from to outputDataSet.
    
    Parameters:
    
    from - the other dataset
  - fetchPrimitives
```
protected void fetchPrimitives(java.util.Set<java.lang.Long> ids,
                               OsmPrimitiveType type,
                               ProgressMonitor progressMonitor)
                        throws OsmTransferException
```
    fetches a set of ids of a given OsmPrimitiveType from the server
    
    Parameters:
    
    ids - the set of ids
    
    type - The primitive type. Must be one of NODE, WAY, RELATION
    
    progressMonitor - progress monitor
    
    Throws:
    
    OsmTransferException - if an error occurs while communicating with the API server
  - parseOsm
```
public DataSet parseOsm(ProgressMonitor progressMonitor)
                 throws OsmTransferException
```
    invokes one or more Multi Gets to fetch the OsmPrimitives and replies the dataset of retrieved primitives. Note that the dataset includes non visible primitives too! In contrast to a simple Get for a node, a way, or a relation, a Multi Get always replies the latest version of the primitive (if any), even if the primitive is not visible (i.e. if visible==false).
    Invoke getMissingPrimitives() to get a list of primitives which have not been found on the server (the server response code was 404)
    
    Specified by:
    
    parseOsm in class OsmServerReader
    
    Parameters:
    
    progressMonitor - progress monitor
    
    Returns:
    
    the parsed data
    
    Throws:
    
    OsmTransferException - if an error occurs while communicating with the API server
    
    See Also:
    
    getMissingPrimitives()
  - checkMissing
```
private void checkMissing(DataSet ds,
                          ProgressMonitor progressMonitor)
                   throws OsmTransferException
```
    Workaround for difference in Overpass API. As of now (version 7.55) Overpass api doesn't return invisible objects. Check if we have objects which do not appear in the dataset and fetch them from OSM instead.
    
    Parameters:
    
    ds - the dataset
    
    progressMonitor - progress monitor
    
    Throws:
    
    OsmTransferException - if an error occurs while communicating with the API server
  - downloadRelations
```
private void downloadRelations(ProgressMonitor progressMonitor)
                        throws OsmTransferException
```
    Finds best way to download a set of relations.
    
    Parameters:
    
    progressMonitor - progress monitor
    
    Throws:
    
    OsmTransferException - if an error occurs while communicating with the API server
    
    See Also:
    
    getMissingPrimitives()
  - getMissingPrimitives
```
public java.util.Set<PrimitiveId> getMissingPrimitives()
```
    replies the set of ids of all primitives for which a fetch request to the server was submitted but which are not available from the server (the server replied a return code of 404)
    
    Returns:
    
    the set of ids of missing primitives
  - setRecurseDownRelations
```
public MultiFetchServerObjectReader setRecurseDownRelations(boolean recurseDownRelations)
```
    Should downloaded relations be complete?
    
    Parameters:
    
    recurseDownRelations - true: yes, recurse down to retrieve the members of the relation This will download sub relations, complete way members and nodes. Members of sub relations are not retrieved unless they are also members of the relations. See #18835.
    
    Returns:
    
    this
    
    Since:
    
    15811
  - setRecurseDownAppended
```
public MultiFetchServerObjectReader setRecurseDownAppended(boolean recurseAppended)
```
    Determine how appended objects are treated. By default, all children of an appended object are also appended.
    
    Parameters:
    
    recurseAppended - false: do not append known children of appended objects, i.e. all nodes of way and all members of a relation
    
    Returns:
    
    this
    
    Since:
    
    15811
  - cancel
```
public void cancel()
```
    Description copied from class: OsmConnection
    
    Cancels the connection.
    
    Overrides:
    
    cancel in class OsmConnection

Class MultiFetchServerObjectReader

Nested Class Summary

Nested classes/interfaces inherited from class org.openstreetmap.josm.io.OsmServerReader

Nested classes/interfaces inherited from class org.openstreetmap.josm.io.OsmConnection

Field Summary

Fields inherited from class org.openstreetmap.josm.io.OsmServerReader

Fields inherited from class org.openstreetmap.josm.io.OsmConnection

Constructor Summary

Method Summary

Methods inherited from class org.openstreetmap.josm.io.OsmServerReader

Methods inherited from class org.openstreetmap.josm.io.OsmConnection

Methods inherited from class java.lang.Object

Field Detail

MAX_IDS_PER_REQUEST

nodes

ways

relations

missingPrimitives

outputDataSet

primitivesMap

recurseDownRelations

recurseDownAppended

exec

Constructor Detail

MultiFetchServerObjectReader

Method Detail

create

create

append

append

appendNode

appendWay

appendRelation

append

append

extractIdPackage

buildRequestString

rememberNodesOfIncompleteWaysToLoad

merge

fetchPrimitives

parseOsm

checkMissing

downloadRelations

getMissingPrimitives

setRecurseDownRelations

setRecurseDownAppended

cancel