問題描述

注意:我在標題中添加了 kif 只是為了搜索索引，考慮到大部分答案都是討論它

我正在為 iOS 尋找類似 selenium 的東西，基本上是一個測試自動化/單元測試框架，它可以多次運行某個 UI 場景直到它崩潰，這將幫助我縮小導致 UI 錯誤的原因非常罕見且隨機地發生.

(順便說一句，我已經對數據源/表交互的每一行代碼進行了 NSLogged 并花費了數小時分析潛在原因……但沒有發現任何結論……再次，這個錯誤很少發生).

我查看了一些 和 KIF.我最終決定使用 KIF 同時借用 cucumber's Gherkin 語法 來描述我的單元測試.

我選擇 KIF(而不是 Frank)的原因是 KIF 是 100% 基于 obj-c，而不是使用 ruby?? 作為Frank 也是如此.所以設置更簡單，更適用于我狹隘的測試用例需求.話雖如此，我承認如果我的應用程序更復雜(即使用來自多個服務器的輸入等)，Frank 會更有用.你可以看到這個優秀的最后一個季度演示文稿以了解更多關于 KIF、Frank 和其他自動化測試框架的優缺點，包括 Apple 自己的 UI 自動化.

使用 KIF 后，我發現了導致上述錯誤的錯誤，并且我可以 100% 使用 KIF 重現它！它很少發生的原因是因為它只有在我非常快地點擊屏幕時才會發生..并且由于 KIF 會自動執行這些步驟..它以非常快的速度執行它們..這暴露了錯誤:).

以下是我用于測試的代碼示例.這只是為了讓您快速了解 KIF(和 Gherkin)可以為您做什么:

在一個文件中我指定了我想要運行的場景:

- (void)initializeScenarios;{[self addScenario:[KIFTestScenario scenarioToCompleteSignInAndLoadInbox]];[self addScenario:[KIFTestScenario scenarioToFillAttachmentsWithData]];[self addScenario:[KIFTestScenario 場景ToViewAndLoadFileBucket]];[self addScenario:[KIFTestScenario 場景ToViewAndLoadFileBucketSubView]];}

每個場景都映射到步驟(要了解更多關于基于測試驅動程序開發的小黃瓜語法和行為驅動開發，我強烈建議閱讀這本關于 黃瓜):

/* @given 應用程序處于全新狀態@并且用戶已經有一個 imap 電子郵件帳戶，該帳戶具有有效的用戶名/密碼@then 用戶可以成功登錄@and 收件箱視圖將被加載@并且收件箱將加載用戶收件箱中的最新一批電子郵件*/+ (id)scenarioToCompleteSignInAndLoadInbox{KIFTestScenario *scenario =[KIFTestScenario scenarioWithDescription:@"測試一個用戶可以成功登錄."];[場景 addStepsFromArray:[KIFTestStep stepsCompleteSignInAndLoadInbox]];返回場景；}/* @假設用戶已經登錄@并且用戶已經下載了他們的文件夾@then 用戶可以點擊文件夾視圖@and 用戶可以點擊附件"遠程文件夾@and 將下載附件"遠程文件夾中的最新批次*/+ (id)scenarioToFillAttachmentsWithData {KIFTestScenario* 場景 =[KIFTestScenario scenarioWithDescription:@"測試我們可以查看附件文件夾并填寫它與數據."];[場景 addStepsFromArray:[KIFTestStep stepsToFillAttachmentsWithData]];返回場景；}/* @假設用戶已經登錄@并且用戶已經下載了他們的文件夾@and 用戶已經下載了附件@then 用戶可以點擊收件箱菜單按鈕@and 用戶可以點擊文件夾列表菜單按鈕@并且用戶可以單擊文件桶圖標(在帳戶列表視圖上)@and 文件桶的數據是從數據庫中獲取的@and 文件桶視圖顯示附件*/+ (id)scenarioToViewAndLoadFileBucket {KIFTestScenario *scenario =[KIFTestScenario scenarioWithDescription:@"測試一個用戶可以成功查看和加載文件桶父視圖"];[場景 addStepsFromArray:[KIFTestStep stepsToViewAndLoadFileBucketPage]];返回場景；}/* @假設用戶已經登錄@并且用戶已經下載了他們的文件夾@and 用戶已經下載了附件@并且用戶已經打開了文件桶視圖@then 用戶可以點擊文件桶視圖表中的隨機行@and 子視圖將從數據庫中檢索與該行相關的數據@and subview 將在 uitableview 中顯示數據*/+ (id)scenarioToViewAndLoadFileBucketSubView {KIFTestScenario *scenario =[KIFTestScenario scenarioWithDescription:@"測試一個用戶可以成功查看和加載文件桶子視圖"];[場景 addStepsFromArray:[KIFTestStep stepsToViewAndLoadFileBucketSubPage]];返回場景；}

使用 KIF 的 UI 自動化方法定義步驟(這只是一個示例):

//此步驟假設有一個附件文件夾，其中包含帶有附件的電子郵件+ (NSArray *)stepsToFillAttachmentsWithData {NSMutableArray* 步驟 = [@[] mutableCopy];【步驟添加對象:[KIFTestStep stepToTapViewWithAccessibilityLabel:@"InboxMenuButton"]];NSIndexPath* indexPath =[NSIndexPath indexPathForRow:remoteAttachmentFolderNumber inSection:0];KIFTestStep* tapAttachmentRowStep =[KIFTestStep stepToTapRowInTableViewWithAccessibilityLabel:@"附件" atIndexPath:indexPath];[步驟添加對象:[KIFTestStep stepToWaitForNotificationName:(NSString *)kBeganSyncingOlderEmails 對象:無whileExecutingStep:tapAttachmentRowStep]];[步驟 addObject:tapAttachmentRowStep];【步驟添加對象:[KIFTestStep stepToWaitForViewWithAccessibilityLabel:@"attachments"]];KIFTestStep *fillingInboxStep =[KIFTestStep stepToWaitForNotificationName:(NSString *)kOldMailBatchDelivered object:nil];[fillingInboxStep setTimeout:kSpecialTimeoutForLongTests];[步驟addObject:fillingInboxStep];返回步驟；}

<小時>

KIF 2.0 示例代碼:KIF 2.0 使用 Xcode 5 的全新 test navigator.. 這是一個巨大 比 KIF 1.0 所做的改進.. 現在您的測試感覺比過去更加有機和自然..(即它是實時進行的..而不是創建在未來運行的場景等)..你甚至可以用播放按鈕等來測試每個..你應該試試看.

這里有一些例子(同樣使用 gherkin 語法):

#import <KIF/KIF.h>#import "KIFUITestActor+EXAdditions.h"#import "KIFUITestActor+UserRegistration.h"@interface 登錄測試:KIFTestCase@結尾@implementation 登錄測試- (void)testReset {[測試儀flushDbase]；[測試儀重置]；}/* @假設應用程序處于全新的干凈狀態@and 沒有人在服務器上注冊過@then 用戶可以在服務器上注冊他們自己@and 立即從騎手的地圖開始@and 他們在地圖上的位置顯示*/- (void)testRegistration{[測試儀flushDbase]；[測試儀重置]；[測試者單用戶注冊];[測試人員 showUserCurrentLocationOnMap];}/* @假設用戶已經注冊到服務器@and 用戶當前沒有登錄@then 用戶可以使用他們的用戶名和密碼登錄@and 立即從騎手的地圖開始@and 他們在地圖上的位置顯示*/- (void)testSuccessfulLogin{[測試儀重置]；[測試人員登錄]；[測試人員 showUserCurrentLocationOnMap];}/* @假設用戶已經注冊@并且用戶在應用程序啟動之前已經登錄@then 用戶從地圖視圖開始，位置可見@and 按鈕提示他們設置接送地點*/- (void)testStartOfApplication {[測試人員 showUserCurrentLocationOnMap];[測試儀顯示PickUpButton]；}@結尾

下面是分類文件中一些測試用例的實現:

- (void)reset{[自我運行塊:^KIFTestStepResult(NSError **錯誤){BOOL 成功重置 = YES;//為您的應用程序執行實際重置.如果失敗，則設置successfulReset = NO.AppDelegate* appDelegate = [[UIApplication sharedApplication] 委托];[appDelegate resetApp];KIFTestCondition(successfulReset, error, @"未能重置應用程序的某些部分.");返回 KIFTestStepResultSuccess；}];}- (void)flushDbase {[自我運行塊:^KIFTestStepResult(NSError **錯誤){NSURL *url = [NSURL URLWithString:@"http://randomdomain.com/flush_db"];NSURLRequest *request = [NSURLRequest requestWithURL:url];NSError *connectionError = nil;BOOL databaseFlushSucceeded = YES;NSURLResponse *響應；NSData *resultData = [NSURLConnection sendSynchronousRequest:request returnedResponse:&response error:&connectionError];如果(！結果數據){databaseFlushSucceeded = 否；KIFTestCondition(databaseFlushSucceeded, error, @"連接服務器失敗！");}如果(連接錯誤){databaseFlushSucceeded = 否；KIFTestCondition(databaseFlushSucceeded, error, [NSString stringWithFormat:@"連接失敗.錯誤: %@", [connectionError localDescription]]);}返回 KIFTestStepResultSuccess；}];}- (void)navigateToLoginPage{[self tapViewWithAccessibilityLabel:@"登錄郵箱"];}- (void)returnToLoggedOutHomeScreen{[self tapViewWithAccessibilityLabel:@"Logout"];[self tapViewWithAccessibilityLabel:@"Logout"];//關閉警報.}

Note: I added kif to the title just for search indexing puposes, considering that most of the answer turned out to discuss it

I'm looking for something like selenium for iOS, basically a test-automation/unit test framework that can run a certain UI scenario many many times until it crashes, which would help me narrow down the cause of a UI bug that happens very rarely and randomly.

(and by the way, I've NSLogged every single line of code of datasource/table interaction and spent hours analyzing the potential cause.. but found nothing conclusive.. again this bug very rarely happens).

I looked at some of the unit testing frameworks in iOS, but they seem to be so many. I'm not sure which to pick. Also my reference to selenium is based on conjecture, as I've worked with QA folks who've used Selenium in large web projects in the past (and i'm assuming that there must be something similar for iOS).

Now that I'm a one man team working on an iOS project, I'm gonna have to put a QA hat on and figure this bug out.

I'm facing a classic bug that happens when there is a discrepancy between the actual number of rows inserted in a UITableView and the number of rows that the datasource delegate returns. This is the error message:

*** Assertion failure in -[UITableView
 _endCellAnimationsWithContext:] Exception in insertRows: Invalid
 update: invalid number of rows in section 0.

The number of rows contained in an existing section after the update (2) must be equal to
 the number of rows contained in that section before the update (2),
 plus or minus the number of rows inserted or deleted from that section
 (1 inserted, 0 deleted) and plus or minus the number of rows moved
 into or out of that section (0 moved in, 0 moved out).

I click on a UITableViewCell that takes me into another UITableView. Sometimes it works

and sometimes (very rarely) it doesn't (with the above error):

update:.. i've added example code about KIF 2.0 at the bottom after divider.. for those who are more interested in KIF than the specific problem i'm facing:

After some research and experimenting.. I've narrowed down my options to two test-automation libraries: Frank and KIF. I ultimately decided to use KIF while borrowing cucumber's Gherkin syntax to describe my unit tests.

The reason I chose KIF (rather than Frank) was that KIF is 100% obj-c based, rather than using ruby as well as was the case with Frank. So setting up is simpler, and it was more applicable to my narrow test case requirement. That being said, I admit Frank would be more useful if my application was more complicated (ie using intput from multiple servers etc). You can see the last quarter of this excellent presentation to learn more about the pros and cons of KIF, Frank and other automation-testing frameworks including Apple's own UI Automation.

After using KIF, I found the bug causing the error above, and I could reproduce it using KIF 100% of the time! The reason why it happened so rarely was because it happened only when I tapped through the screens really fast.. and since KIF automates the steps.. it does them at an incredibly fast speed.. which exposed the bug :).

So following will be a sample of the code I used for testing.. this is just to give you a quick feel of what KIF (and Gherkin) can do for you:

in one file I specify the scenarios I want to run:

- (void)initializeScenarios;
{
    [self addScenario:[KIFTestScenario scenarioToCompleteSignInAndLoadInbox]];
    [self addScenario:[KIFTestScenario scenarioToFillAttachmentsWithData]];
    [self addScenario:[KIFTestScenario scenarioToViewAndLoadFileBucket]];
    [self addScenario:[KIFTestScenario scenarioToViewAndLoadFileBucketSubView]];
}

each scenario maps to steps (to understand more about the gherkin syntax -and behavioral driven development, which is based on test driver development, I strongly recommend to read this excellent book about cucumber):

/* @given the application is at a fresh state
   @and   the user already has an imap email account with a valid username/pwd

   @then  the user can successfully log in
   @and   the inbox view will be loaded
   @and   the inbox will get loaded with the latest batch of emails in the user inbox
 */
+ (id)scenarioToCompleteSignInAndLoadInbox
{
    KIFTestScenario *scenario = 
      [KIFTestScenario scenarioWithDescription:@"Test that a user 
                                                 can successfully log in."];
    [scenario addStepsFromArray:[KIFTestStep stepsCompleteSignInAndLoadInbox]];

    return scenario;
}


/* @given that the user is already signed in
   @and   the user has already downloaded their folders 

   @then  the user can click on the folders view
   @and   the user can click on the 'attachments' remote folder
   @and   the latest batch from the 'attachments' remote folder will download
 */
+ (id)scenarioToFillAttachmentsWithData {
    KIFTestScenario* scenario = 
      [KIFTestScenario scenarioWithDescription:@"Test that we can view the 
                                                 attachments folder and fill 
                                                 it with data."];
    [scenario addStepsFromArray:[KIFTestStep stepsToFillAttachmentsWithData]];
    return scenario;

}

/* @given that the user is already signed in
   @and   the user has already downloaded their folders
   @and   the user has already downloaded attachments

   @then  the user can click on inbox menu button
   @and   the user can click on folder list menu button
   @and   the user can click on the file bucket icon (on the account list view)
   @and   the data for the file bucket is fetched from the dbase
   @and   the file bucket view displayes the attachments
 */
+ (id)scenarioToViewAndLoadFileBucket {
    KIFTestScenario *scenario = 
       [KIFTestScenario scenarioWithDescription:@"Test that a user can successfully 
                                                  view and load 
                                                  file bucket parent view"];
    [scenario addStepsFromArray:[KIFTestStep stepsToViewAndLoadFileBucketPage]];

    return scenario;
}

/* @given that the user is already signed in
   @and   the user has already downloaded their folders
   @and   the user has already downloaded attachments
   @and   the user has already opened file bucket view 

   @then  the user can click on a random row in the file bucket view table
   @and   the subview will retrieve data from the dbase pertaining to that row
   @and   the subview will display the data in the uitableview
 */
+ (id)scenarioToViewAndLoadFileBucketSubView {
    KIFTestScenario *scenario = 
       [KIFTestScenario scenarioWithDescription:@"Test that a user can successfully
                                                  view and load filet
                                                  bucket sub view"];
    [scenario addStepsFromArray:[KIFTestStep stepsToViewAndLoadFileBucketSubPage]];
    return scenario;   
}

and steps are defined using KIF's UI automation methods (this is just one example):

// this step assumes there is an attachment folder that contains emails with attachments
+ (NSArray *)stepsToFillAttachmentsWithData {

    NSMutableArray* steps = [@[] mutableCopy];

    [steps addObject:
        [KIFTestStep stepToTapViewWithAccessibilityLabel:@"InboxMenuButton"]];

    NSIndexPath* indexPath = 
        [NSIndexPath indexPathForRow:remoteAttachmentFolderNumber inSection:0];
    KIFTestStep* tapAttachmentRowStep = 
        [KIFTestStep stepToTapRowInTableViewWithAccessibilityLabel:
                                     @"attachments" atIndexPath:indexPath];

    [steps addObject:[KIFTestStep stepToWaitForNotificationName:
         (NSString *)kBeganSyncingOlderEmails object:nil           
                          whileExecutingStep:tapAttachmentRowStep]];

    [steps addObject:tapAttachmentRowStep];

    [steps addObject:
        [KIFTestStep stepToWaitForViewWithAccessibilityLabel:@"attachments"]];

    KIFTestStep *fillingInboxStep = 
        [KIFTestStep stepToWaitForNotificationName:
                                 (NSString *)kOldMailBatchDelivered object:nil];

    [fillingInboxStep setTimeout:kSpecialTimeoutForLongTests];
    [steps addObject:fillingInboxStep];

    return steps;
}

KIF 2.0 sample code: KIF 2.0 uses Xcode 5's all new test navigator.. which is a huge improvement than what KIF 1.0 was doing.. now your tests feel a lot more organic and natural than the past.. (ie it goes in real time.. rather than creating scenarios that run in the future etc).. you even get to test each one with a play button etc.. you should try it out.

here are some examples (again using gherkin syntax):

#import <KIF/KIF.h>
#import "KIFUITestActor+EXAdditions.h"
#import "KIFUITestActor+UserRegistration.h"

@interface LoginTests : KIFTestCase

@end
@implementation LoginTests

- (void)testReset {
    [tester flushDbase];
    [tester reset];
}

/* @given that the app is in a fresh clean state
 @and   that no one has ever registered with the server

 @then  the user can register their themselves with the server
 @and   immediately start with the rider's map
 @and   their location on the map shows
 */

- (void)testRegistration
{
    [tester flushDbase];
    [tester reset];
    [tester singleUserRegistration];
    [tester showUserCurrentLocationOnMap];
}

/* @given that the user has already registered with the server
   @and the user is not currently logged in

 @then  the user can login using their user name and password
 @and   immediately start with the rider's map
 @and   their location on the map shows
 */
- (void)testSuccessfulLogin
{
    [tester reset];
    [tester login];
    [tester showUserCurrentLocationOnMap];
}

/* @given that the user has already registered 
   @and that the user is already logged in before app launch

 @then the user starts on the map view with the location visible
 @and the button prompts them to set pick up location
 */
- (void)testStartOfApplication {
    [tester showUserCurrentLocationOnMap];
    [tester showsPickUpButton];
}
@end

here is the implementation of some of the test cases in the category files:

- (void)reset
{
    [self runBlock:^KIFTestStepResult(NSError **error) {
        BOOL successfulReset = YES;

        // Do the actual reset for your app. Set successfulReset = NO if it fails.
        AppDelegate* appDelegate = [[UIApplication sharedApplication] delegate];
        [appDelegate resetApp];

        KIFTestCondition(successfulReset, error, @"Failed to reset some part of the application.");

        return KIFTestStepResultSuccess;
    }];
}

- (void)flushDbase {
    [self runBlock:^KIFTestStepResult(NSError **error){
        NSURL *url = [NSURL URLWithString:@"http://randomdomain.com/flush_db"];
        NSURLRequest *request = [NSURLRequest requestWithURL:url];
        NSError *connectionError = nil;

        BOOL databaseFlushSucceeded = YES;

        NSURLResponse *response;
        NSData *resultData = [NSURLConnection sendSynchronousRequest:request returningResponse:&response error:&connectionError];
        if (!resultData) {
            databaseFlushSucceeded = NO;
            KIFTestCondition(databaseFlushSucceeded, error, @"failed to connect to server!");
        }

        if (connectionError) {
            databaseFlushSucceeded = NO;
            KIFTestCondition(databaseFlushSucceeded, error, [NSString stringWithFormat:@"connection failed. Error: %@", [connectionError localizedDescription]]);
        }

        return KIFTestStepResultSuccess;
    }];
}


- (void)navigateToLoginPage
{
    [self tapViewWithAccessibilityLabel:@"login email"];
}

- (void)returnToLoggedOutHomeScreen
{
    [self tapViewWithAccessibilityLabel:@"Logout"];
    [self tapViewWithAccessibilityLabel:@"Logout"]; // Dismiss alert.
}

這篇關于KIF:如何自動運行/壓力測試 iOS 應用程序以找出罕見 UI 錯誤的原因?的文章就介紹到這了，希望我們推薦的答案對大家有所幫助，也希望大家多多支持html5模板網！