//
// Licensed to the Apache Software Foundation (ASF) under one
// or more contributor license agreements.  See the NOTICE file
// distributed with this work for additional information
// regarding copyright ownership.  The ASF licenses this file
// to you under the Apache License, Version 2.0 (the
// "License"); you may not use this file except in compliance
// with the License.  You may obtain a copy of the License at
//
//   http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing,
// software distributed under the License is distributed on an
// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
// KIND, either express or implied.  See the License for the
// specific language governing permissions and limitations
// under the License.
//
[[provisioning-pull]]
==== Pull

Pull is the mechanism used to acquire identity data from Identity Stores; for each external resource, one or more
<<tasks-pull,pull tasks>> can be defined, run and scheduled for period execution.

Pull task execution involves querying the external resource for all <<mapping,mapped>> <<anytype, any types>>, sorted
according to the order defined by a custom implementation of
ifeval::["{snapshotOrRelease}" == "release"]
https://github.com/apache/syncope/blob/syncope-{docVersion}/core/provisioning-api/src/main/java/org/apache/syncope/core/provisioning/api/ProvisionSorter.java[ProvisionSorter^]
endif::[]
ifeval::["{snapshotOrRelease}" == "snapshot"]
https://github.com/apache/syncope/blob/4_1_X/core/provisioning-api/src/main/java/org/apache/syncope/core/provisioning/api/ProvisionSorter.java[ProvisionSorter^]
endif::[]
or its default implementation
ifeval::["{snapshotOrRelease}" == "release"]
https://github.com/apache/syncope/blob/syncope-{docVersion}/core/provisioning-java/src/main/java/org/apache/syncope/core/provisioning/java/pushpull/DefaultProvisionSorter.java[DefaultProvisionSorter^]
endif::[]
ifeval::["{snapshotOrRelease}" == "snapshot"]
https://github.com/apache/syncope/blob/4_1_X/core/provisioning-java/src/main/java/org/apache/syncope/core/provisioning/java/pushpull/DefaultProvisionSorter.java[DefaultProvisionSorter^]
endif::[]
.

Each entity is then processed in an isolated transaction; a retrieved entity can be:

. _matching_ if a corresponding internal entity was found, according to the <<mapping,mapping>> of - or the
<<policies-inbound,inbound policy>> set for, if present - the enclosing external resource;
. _unmatching_ otherwise.

Once this has been assessed, entities are processed according to the matching / unmatching rules specified for the pull task:
by default, unmatching entities get created internally, and matching entities are updated.

.Matching Rules
****
* `IGNORE`: do not perform any action;
* `UPDATE`: update matching entity;
* `DEPROVISION`: delete external entity;
* `UNLINK`: remove association with external resource, without performing any (de-)provisioning operation;
* `LINK`: associate with external resource, without performing any (de-)provisioning operation;
* `UNASSIGN`: unlink and delete.
****

.Unmatching Rules
****
* `IGNORE`: do not perform any action;
* `UNLINK`: do not perform any action;
* `ASSIGN`: create internally, assign the external resource;
* `PROVISION`: create internally, do not assign the external resource.
****

[[pull-mode]]
[TIP]
.Pull Mode
====
The Identity Store can be queried in different ways, depending on the _pull mode_ that is specified:

****
FULL RECONCILIATION:: The complete list of entities available is processed.
FILTERED RECONCILIATION:: The subset matching the filter (provided by the selected implementation of
ifeval::["{snapshotOrRelease}" == "release"]
https://github.com/apache/syncope/blob/syncope-{docVersion}/core/provisioning-api/src/main/java/org/apache/syncope/core/provisioning/api/pushpull/ReconFilterBuilder.java[ReconFilterBuilder^])
endif::[]
ifeval::["{snapshotOrRelease}" == "snapshot"]
https://github.com/apache/syncope/blob/4_1_X/core/provisioning-api/src/main/java/org/apache/syncope/core/provisioning/api/pushpull/ReconFilterBuilder.java[ReconFilterBuilder^])
endif::[]
of all available entities is processed.
INCREMENTAL:: Only the actual modifications performed since the last pull task execution are considered. This mode
requires the underlying connector bundle to implement the ConnId `SYNC` operation - only some of the available bundles
match this condition. +
*This is the only mode which allows pulling delete events*, which may end up causing the removal of internal entities.
****
====

[[pull-templates]]
[TIP]
.Pull Templates
====
With every <<tasks-pull,pull task>> it is possible to add a template for each defined <<anytype,any type>>.

As the values specified in the template are applied to pulled entities, this can be used as mechanism for setting
default values for attributes or external resources on entities.

A typical use case is, when pulling Users from the external resource `R`, to automatically assign `R` so that every
further modification in Apache Syncope to such Users will be <<propagation,propagated>> back to `R`.
====

===== InboundActions

The pull process can be decorated with custom logic to be invoked around task execution, by associating
pull tasks to one or more <<implementations,implementations>> of the
ifeval::["{snapshotOrRelease}" == "release"]
https://github.com/apache/syncope/blob/syncope-{docVersion}/core/provisioning-api/src/main/java/org/apache/syncope/core/provisioning/api/pushpull/InboundActions.java[InboundActions^]
endif::[]
ifeval::["{snapshotOrRelease}" == "snapshot"]
https://github.com/apache/syncope/blob/4_1_X/core/provisioning-api/src/main/java/org/apache/syncope/core/provisioning/api/pushpull/InboundActions.java[InboundActions^]
endif::[]
interface.

Some examples are included by default, see the table below.

[cols="1,2"]
|===

|
ifeval::["{snapshotOrRelease}" == "release"]
https://github.com/apache/syncope/blob/syncope-{docVersion}/core/provisioning-java/src/main/java/org/apache/syncope/core/provisioning/java/pushpull/ADMembershipPullActions.java[ADMembershipPullActions^]
endif::[]
ifeval::["{snapshotOrRelease}" == "snapshot"]
https://github.com/apache/syncope/blob/4_1_X/core/provisioning-java/src/main/java/org/apache/syncope/core/provisioning/java/pushpull/ADMembershipPullActions.java[ADMembershipPullActions^]
endif::[]
| If a User is associated with a Group in AD, keep the corresponding User as a member of the corresponding Group in Syncope.

| 
ifeval::["{snapshotOrRelease}" == "release"]
https://github.com/apache/syncope/blob/syncope-{docVersion}/core/provisioning-java/src/main/java/org/apache/syncope/core/provisioning/java/pushpull/LDAPMembershipPullActions.java[LDAPMembershipPullActions^]
endif::[]
ifeval::["{snapshotOrRelease}" == "snapshot"]
https://github.com/apache/syncope/blob/4_1_X/core/provisioning-java/src/main/java/org/apache/syncope/core/provisioning/java/pushpull/LDAPMembershipPullActions.java[LDAPMembershipPullActions^]
endif::[]
| If a User is associated with a Group in LDAP, keep the corresponding User as a member of the corresponding Group in Syncope.

| 
ifeval::["{snapshotOrRelease}" == "release"]
https://github.com/apache/syncope/blob/syncope-{docVersion}/core/provisioning-java/src/main/java/org/apache/syncope/core/provisioning/java/pushpull/LDAPPasswordPullActions.java[LDAPPasswordPullActions^]
endif::[]
ifeval::["{snapshotOrRelease}" == "snapshot"]
https://github.com/apache/syncope/blob/4_1_X/core/provisioning-java/src/main/java/org/apache/syncope/core/provisioning/java/pushpull/LDAPPasswordPullActions.java[LDAPPasswordPullActions^]
endif::[]
| Import hashed password values from LDAP;
the cipher algorithm associated with the password must match the value of `passwordHashAlgorithm` for the 
https://connid.atlassian.net/wiki/display/BASE/LDAP#LDAP-Configuration[LDAP connector bundle^].

| 
ifeval::["{snapshotOrRelease}" == "release"]
https://github.com/apache/syncope/blob/syncope-{docVersion}/core/provisioning-java/src/main/java/org/apache/syncope/core/provisioning/java/pushpull/DBPasswordPullActions.java[DBPasswordPullActions^]
endif::[]
ifeval::["{snapshotOrRelease}" == "snapshot"]
https://github.com/apache/syncope/blob/4_1_X/core/provisioning-java/src/main/java/org/apache/syncope/core/provisioning/java/pushpull/DBPasswordPullActions.java[DBPasswordPullActions^]
endif::[]
| Import hashed password values from DBMS;
the cipher algorithm associated with the password must match the value of `Password cipher algorithm` for the 
https://connid.atlassian.net/wiki/display/BASE/Database+Table#DatabaseTable-ConfigurationProperties[DatabaseTable connector bundle^].

| 
ifeval::["{snapshotOrRelease}" == "release"]
https://github.com/apache/syncope/blob/syncope-{docVersion}/core/provisioning-java/src/main/java/org/apache/syncope/core/provisioning/java/pushpull/KafkaInboundActions.java[KafkaInboundActions^]
endif::[]
ifeval::["{snapshotOrRelease}" == "snapshot"]
https://github.com/apache/syncope/blob/4_1_X/core/provisioning-java/src/main/java/org/apache/syncope/core/provisioning/java/pushpull/KafkaInboundActions.java[KafkaInboundActions^]
endif::[]
| Instructs to fetch the attributes required during the <<provisioning-livesync,live sync>> process for the 
https://connid.atlassian.net/wiki/display/BASE/Kafka[Apache Kafka connector bundle^].

|===

===== Remediation

Errors during pull might arise for various reasons: values might not be provided for all mandatory attributes or 
fail the configured validation, and so on.

When Remediation is enabled for a certain <<tasks-pull,Pull Task>>, execution errors are reported to administrators,
which are given the chance to examine and possibly fix, or just discard.
