BitBar/BrowserStack: When("I $actionType baseline with name $name ignoring:$checkSettings") step take image from a browser haeader instead of page itself

[AntonD2203]

STR:

1. Run test.story on iOS BitBar (may be any available iOS device) device:

Scenario: Take a baseline
Given I am on a page with the URL 'https://github.com/'
When I change context to element located `By.xpath(//header)`
When I take screenshot
When I COMPARE_AGAINST baseline with name `name` ignoring:
|ACCEPTABLE_DIFF_PERCENTAGE	|
|15							        |

2. Verify results for Checpoint image from When("I $actionType baseline with name $name ignoring:$checkSettings") step

Applied configuration:

web-application.sticky-header-size-percentage=40
selenium.grid.enabled=true
selenium.grid.url=https://eu-mobile-hub.bitbar.com/wd/hub
bdd.variables.global.platform=${selenium.grid.capabilities.platformName}
!-- Please set up your own API key in overriding properties so that you can access run details, video, etc
selenium.grid.capabilities.bitbar_apiKey=
!-- Capability prevents BitBar to use similar devices in case of unavailability of the selected device
selenium.grid.capabilities.bitbar_findDevice=true
!-- Capability that forces BitBar to remove files created during the testing (prevents session retention that could be visible to other users)
selenium.grid.capabilities.clearSystemFiles=True
selenium.grid.capabilities.platformName=iOS
selenium.grid.capabilities.deviceName=iPhone device
selenium.grid.capabilities.automationName=XCUITest
selenium.grid.capabilities.browserName=Safari
selenium.grid.capabilities.bitbar_device=Apple iPad Pro A1584 15.3

Expected result:
Checpoint image is taken from part of the page which is in the context
Actual result:
Checpoint image is taken from part of the Safari browser header

NOTE:
The same issue affects BrowserStack iOS devices:
allure report -
allure.zip
session URL ([email protected] has access):
https://automate.browserstack.com/dashboard/v2/builds/44a2686f272e989fac7e2e533797d8bba6b1cb1f/sessions/66a3ee581005aea29a9aca720d88d2f0cfca2ec6?auth_token=4b802276457b0d1d85def613488f09d7acbdb90fe74e565f59fc58554daf30c4

Device capabilities:
Requested capabilities:

{
  "appiumVersion" : "1.19.1",
  "automationName" : "XCUITest",
  "bitbar_apiKey" : "IEtphJM8Vd88OdgaeyRVKvLgiE5GfVLE",
  "bitbar_device" : "Apple iPad Pro A1584 15.3",
  "bitbar_find
  "Device" : true,
  "bitbar_project" : "Saturn",
  "bitbar_testTimeout" : "1200",
  "browserName" : "Safari",
  "clearSystemFiles" : true,
  "deviceName" : "iPhone device",
  "deviceOrientation" : "portrait",
  "platformName" : "IOS",
  "screenshotWaitTimeout" : "20"
}

Session capabilities:

{
  "appium:networkConnectionEnabled" : false,
  "appium:showXcodeLog" : true,
  "appium:clearSystemFiles" : true,
  "appium:takesScreenshot" : true,
  "appium:appiumVersion" : "1.19.1",
  "appium:autoDismissAlerts" : true,
  "appium:automationName" : "XCUITest",
  "appium:locationContextEnabled" : false,
  "appium:screenshotWaitTimeout" : "20",
  "appium:databaseEnabled" : false,
  "appium:udid" : "00008110-001A69502E05801E",
  "appium:wdaLaunchTimeout" : 180000,
  "appium:xcodeConfigFile" : "/opt/testdroid/appium.xcconfig",
  "appium:deviceOrientation" : "portrait",
  "browserName" : "safari",
  "appium:webStorageEnabled" : false,
  "appium:deviceName" : "iPhone device",
  "appium:javascriptEnabled" : true,
  "platformName" : "IOS",
  "appium:usePrebuiltWDA" : true
}

Artifacts:
allure report -
allure.zip
bitbar logs -
bit_bar_logs.zip
Recorded session -
https://user-images.githubusercontent.com/66725458/218119833-5b775844-7feb-40bd-95bd-178e7ffd084b.mp4
Session URL ([email protected] has access) - https://cloud.bitbar.com/#testing/device-session/207165301/209105697/8851070

[AntonD2203]

Here is also results for scenario:

Scenario: Take a baseline
Given I am on a page with the URL 'https://github.com/'
When I COMPARE_AGAINST baseline with name `name` ignoring:
|ACCEPTABLE_DIFF_PERCENTAGE	|
|15							|

allure report -
allure.zip
session URL ([email protected] has access):
https://automate.browserstack.com/dashboard/v2/builds/44a2686f272e989fac7e2e533797d8bba6b1cb1f/sessions/2f56f2b1e529a4df46aa6d45fadeb41a980a1756?auth_token=87c98d2c8aea43303db5d8921678a46298d9d38e1278808fb1f4879319381bbf

[valfirst]

@AntonD2203 is issue really reproduced on BrowserStack?

[valfirst]

@AntonD2203

"appiumVersion" : "1.19.1",

is there any reason to use so old Appium version?

[AntonD2203]

@AntonD2203

"appiumVersion" : "1.19.1",

is there any reason to use so old Appium version?

the version is determined by the bitbar and cannot be changed by us (the corresponding capability will be ignored on startup)
In session logs version specified as [Appium] Welcome to Appium v1.22.2

[AntonD2203]

@AntonD2203 is issue really reproduced on BrowserStack?

yes, there is the report in NOTE section

[AntonD2203]

The bug is reproducible on v. 0.5.1

[valfirst]

Would you mind upgrading to VIVIDUS 0.5.4? Also information about used profiles will be useful

[AntonD2203]

Applied overriding properties:
bdd.story-loader.batch-1.resource-location=story
bdd.story-loader.batch-1.resource-include-patterns=test.story
configuration.profiles=browserstack/ios/iPad9th/landscape,browserstack/web,web/tablet
configuration.environments=whitesite/uat
web-application.sticky-header-size-percentage=40
bdd.configuration.composite-paths=steps/*.steps
bdd.configuration.formats=JSON
known-issue-provider.fileName=known-issues.json
known-issue-provider.detect-potentially-known-issues=false

Applied configuration.profiles properties:
selenium.grid.enabled=true
selenium.grid.username=
selenium.grid.password=
bdd.variables.global.platform=${selenium.grid.capabilities.platformName}
selenium.grid.capabilities.platformName=iOS
bdd.all-meta-filters=groovy: !skip && (${bdd.meta-filters})
selenium.grid.capabilities.browserName=Safari
selenium.grid.capabilities.bstack:options.os=iOS
selenium.grid.capabilities.bstack:options.osVersion=15
selenium.grid.capabilities.bstack:options.realMobile=true
selenium.grid.capabilities.bstack:options.deviceName=iPad 9th
selenium.grid.capabilities.bstack:options.local=false
selenium.grid.capabilities.bstack:options.appiumVersion=1.22.0
bdd.variables.global.layout=tablet_large
selenium.grid.capabilities.bstack:options.deviceOrientation=landscape

Applied configuration.environments properties:
template-processor.resolve-bdd-variables=true

[AntonD2203]

Would you mind upgrading to VIVIDUS 0.5.4? Also information about used profiles will be useful

Will check it

[AntonD2203]

Would you mind upgrading to VIVIDUS 0.5.4? Also information about used profiles will be useful

I've created a test repository to reproduce the bug. v0.5.4. is also affected - https://github.com/AntonD2203/bug_3629_vividus