WWDC18 What's New in LLVM

Preface

The importance of LLVM as the compiling infrastructure for Apple s queen is self-evident. Apple has never stopped maintaining and updating LLVM, and almost every year WWDC has a special session to introduce and explain the new features of LLVM. The just past WWDC18 is no exception.

WWDC18 Session 409 What's New in LLVM in Apple's engineers and introduce us to the LLVM latest features, this article will be combined WWDC18 Session 409 given official presentation to share new features LLVM and I talk about myself personally Humble opinion of these characteristics.

Note: This article will not translate the official presentation word by word, nor will it introduce the basic knowledge of LLVM too much.

index

ARC update
Xcode 10 new diagnostics
Clang static analysis
Increase safety
New instruction set extension
summary

ARC update

The highlight of this ARC update is that ARC Objective-C objects are allowed in C struct.

In the previous version of Xcode, when trying to use Obj-C objects in the definition of C struct, the compiler will throw Error: ARC forbids Objective-C objects in struct , as shown in the following figure:

Well~ This is because LLVM did not support it before. If you write the same code in Xcode 10, there will be no Warning and Error:

So if you use Objective-C objects directly in C struct, is there no memory problem? When is the memory space occupied by Objective-C destroyed?

//ARC Object Pointers in C Structs!
typedef struct {
	NSString *name;
	NSNumber *price;
} MenuItem;

void orderFreeFood(NSString *name) {
	MenuItem item = {
		name,
		[NSNumber numberWithInt:0]
	};
	//[item.name retain];
	//[item.price retain];
	orderMenuItem(item);
	//[item.name release]; 
	//[item.price release];
}

As shown in the above code, the compiler will C struct MenuItemafter creating retainwherein the ARC Objective-C object and orderMenuItem(item);after the statement, i.e., the use of other MenuItem itemafter the function calls releaseout relevant ARC Objective-C object.

Think about it, when it comes to dynamic memory management , what is the difference between the memory management of ARC Objective-C objects?

Note: Dynamic memory management (Dynamic Memory Management), refers to non- int a[100];or MenuItem item = {name, [NSNumber numberWithInt:0]};that after the use of which determines the storage structure, the code will automatically determine the scope and storage period, the code must be subject to pre-established memory management rules.

We know that if you want to flexibly create a dynamically sized array in the C language, you need to manually open up, manage, and release related memory. Example:

void foo() {
	int max;
	double *ptd;
	    
	puts("What is the maximum number of type double entries?");
	scanf("%d", &max);
	ptd = malloc(max * sizeof(double));
	if (ptd == NULL) {
	   //memory allocation failed
	    ...
	}
	    
	//some logic
	...
	
	free(ptd);
}

So should the dynamic memory management of ARC Objective-C in C struct be written like this?

//Structs with ARC Fields Need Care for Dynamic Memory Management
typedef struct {
	NSString *name;
	NSNumber *price;
} MenuItem;

void testMenuItems() {
	//Allocate an array of 10 menu items
	MenuItem *items = malloc(10 * sizeof(MenuItem));
	orderMenuItems(items, 10);
	free(items);
}

The answer is no !

Can be seen through mallocC struct memory initialization opened with ARC Objective-C in ARC Objective-C pointer is not zero-initialized.

Well~ this time I naturally think of using it calloc^_^

Note: callocand malloccan be completed the memory allocation, except that the callocblock of memory will be allocated over all locations are set to 0 in (Note, however, in some hardware systems, floating point value 0 not all 0 bits represented ).

Another problem is that free(items);before the statement is executed, ARC Objective-C has not been cleaned up.

Emmmmm ... official recommendation was written in a free(items);previously itemsall within the struct into the ARC Objective-C pointer manual jobs nil...

So in dynamic memory management , the above code should be written like this:

//Structs with ARC Fields Need Care for Dynamic Memory Management
typedef struct {
	NSString *name;
	NSNumber *price;
} MenuItem;

void testMenuItems() {
	//Allocate an array of 10 menu items
	MenuItem *items = calloc(10, sizeof(MenuItem));
	orderMenuItems(items, 10);
	//ARC Object Pointer Fields Must be Cleared Before Deallocation
	for (size_t i = 0; i < 10; ++i) {
		items[i].name = nil;
		items[i].price = nil;
	}
	free(items);
}

In an instant , there is a feeling of being a dog ? Is there anything wrong with it?

personal opinion

Well~ The addition of support for ARC Objective-C object fields in C struct means that we can build cross-language interactive operations in Objective-C in the future .

Note: The official statement has made some adjustments to the Objective-C++ ABI in order to unify ARC and manual retain/release (MRR) of some functions under the transfer by value and return struct.

It is worth mentioning that Swift does not support this feature (2333~ who said that Objective-C updates are all to cater to the changes in Swift).

Xcode 10 new diagnostics

Swift and Objective-C interoperability

We all know that Swift and Objective-C have a certain degree of interoperability, that is, Swift and Objective-C can be mixed. During the mixing, Xcode generates a header file to expose part of the interface that Swift can be converted into Objective-C.

However, due to the compatibility of Swift and Objective-C, part of the code implemented in Swift cannot be converted to Objective-C.

In recent years, LLVM has consistently tried to make these two languages more interoperable (this is the reason why the update of Objective-C mentioned above is to cater to Swift), this time LLVM supports the closure of Swift. The package (Closures) is imported into Objective-C .

@objc protocol Executor {
	func performOperation(handler: () -> Void)
}

#import  Executor-Swift.h 
@interface DispatchExecutor : NSObject<Executor>
- (void)performOperation:(void (^)(void))handler; 
@end

Note: Closures in Swift are non-escaping closures by default, that is, closures should not be executed after the function returns.

The counterpart of Swift closure in Objective-C is Block, but the Block in Objective-C does not have restrictions such as whether it can escape or not in Swift, so we will turn Swift's non-escaping closures into Objective-C without restrictions. Will there be a problem with the Block?

Don't worry, the converted closure (non-escaping) will have a Warnning prompt, and we said that in this case, Apple engineers will add a macro to Objective-C in LLVM to cater to Swift...

//Warning for Missing Noescape Annotations for Method Overrides
#import  Executor-Swift.h 
@interface DispatchExecutor : NSObject<Executor>
- (void)performOperation:(NS_NOESCAPE void (^)(void))handler;
@end
@implementation DispatchExecutor
- (void)performOperation:(NS_NOESCAPE void (^)(void))handler {
}
//Programmer must ensure that handler is not called after performOperation returns
@end

personal opinion

If Swift 5 can really achieve ABI stability, then the app package size mixed with Swift and Objective-C should also return to normal. I believe that many companies' projects will slowly shift from Objective-C to Swift. The existence of Closures as a first-class citizen in Swift has laid the foundation of Swift as a functional language. This time LLVM provides support for interoperability and conversion between Closures in Swift and Block in Objective-C. It is undoubtedly necessary of.

Use `#pragma pack`packaged Struct members

Emmmmm... To be honest, the content of this section is more low-level, so it may be more obscure, I hope I can express it clearly. In the C language struct with a memory layout (memory layout) concept, each of the C language allows the compiler to specify some basic types of alignment , usually based on the size of the type of standard alignment, but it is specific to the realization of.

Well~ Let s give an example, just take the official presentation from WWDC18:

struct Struct { 
	uint8_t a, b;
	//2 byte padding 
	uint32_t c;
};

In the above example, the compiler to have to align the memory layout Structis inserted between the second two byte field of the third field.

|   1   |   2   |   3   |   4   |
|   a   |   b   | pad.......... |
|  c(1) |  c(2) |  c(3) |  c(4) |

In this way, the struct that should occupy 6 bytes occupies 8 bytes, even though there are only 6 bytes of data.

The C language allows every remote modern compiler to implement #pragma packit, which allows the programmer to control the padding to comply with the ABI.

From C99 6.7.2.1:

12 Each non-bit-field member of a structure or union object is aligned in an implementation- defined manner appropriate to its type.

13 Within a structure object, the non-bit-field members and the units in which bit-fields reside have addresses that increase in the order in which they are declared. A pointer to a structure object, suitably converted, points to its initial member (or if that member is a bit-field, then to the unit in which it resides), and vice versa. There may be unnamed padding within a structure object, but not at its beginning.

In fact on #pragma packthe relevant information can be found in MSDN page found in.

The LLVM also joined to #pragma packsupport, use the following:

#pragma pack (push, 1) 
struct PackedStruct {
	uint8_t a, b;
	uint32_t c; 
};
#pragma pack (pop)

After #pragma packfollowing our struct alignment as follows:

|   1   |
|   a   | 
|   b   |
|  c(1) |
|  c(2) |
|  c(3) |
|  c(4) |

In fact, #pragma pack (push, 1)in 1that alignment byte number, if set 4then the alignment they become back to the original state:

|   1   |   2   |   3   |   4   |
|   a   |   b   | pad.......... |
|  c(1) |  c(2) |  c(3) |  c(4) |

It is worth mentioning that if you use the #pragma pack (push, n)following forget to write #pragma pack (pop)the words, Xcode 10 throws warning:

personal opinion

Well ~ struct When the transmission at the network level, the #pragma packsavings more traffic to a user-defined memory layout alignment.

Clang static analysis

Xcode has always provided a static analyzer (Static Analyzer). Using Clang Static Analyzer can help us find boundary conditions and bugs that are difficult to detect.

Click Product -> Analyze or use the shortcut key Shift+Command+B to statically analyze the currently built project. Of course, you can also automatically perform static analysis when building the project in the project's Build Settings (personally not recommended):

The local static analyzer has the following improvements:

GCD performance anti-pattern
The auto-release variable exceeds the auto-release pool
Improved performance and visualization reports

GCD performance anti-pattern

In some previous unavoidable circumstances, we may need to use the GCD signal ( dispatch_semaphore_t) to block some asynchronous operations, and return the final result obtained after blocking synchronously:

__block NSString *taskName = nil;
dispatch_semaphore_t sema = dispatch_semaphore_create(0);
[self.connection.remoteObjectProxy requestCurrentTaskName:^(NSString *task) {
	taskName = task;
	dispatch_semaphore_signal(sema);
}];
dispatch_semaphore_wait(sema, DISPATCH_TIME_FOREVER);
return taskName;

Well~ what's the problem with writing like this?

The above code exists to block the current thread by using asynchronous threads to execute tasks, and the Task queue usually has a lower priority, so it will cause priority inversion .

So what should we write after Xcode 10?

__block NSString *taskName = nil;
id remoteObjectProxy = [self.connection synchronousRemoteObjectProxyWithErrorHandler:
	^(NSError *error) { NSLog(@"Error: %@", error); }];
[remoteObjectProxy requestCurrentTaskName:^(NSString *task) {
	taskName = task; 
}];
return taskName;

If possible, try to use the synchronousversion of the API. Or, use the asynchronousmethod API:

[self.connection.remoteObjectProxy requestCurrentTaskName:^(NSString *task) { 
	completionHandler(task);
}];

The static analysis check of GCD performance anti-pattern can be enabled under build settings:

The auto-release variable exceeds the auto-release pool

As we all know, the use of __autoreleasingmodifier modified variables will be released when the autorelease pool left (release):

@autoreleasepool {
	__autoreleasing NSError *err = [NSError errorWithDomain:@"domain" code:1 userInfo:nil];
}

This kind of point that does not need our attention is often a hidden danger that causes the program to crash:


- (void)findProblems:(NSArray *)arr error:(NSError **)error {
	[arr enumerateObjectsUsingBlock:^(id value, NSUInteger idx, BOOL *stop) {
		if ([value isEqualToString:@"problem"]) { 
			if (error) {
				*error = [NSError errorWithDomain:@"domain" code:1 userInfo:nil];
			}
		}
	}];
}

Well~ the above code will cause Crash, can you point out why?

Objective-C implicitly used in ARC (Automatic Reference Counting) __autoreleasingmodified error, that is NSError *__autoreleasing*. The -enumerateObjectsUsingBlock:internal iterations will blockuse @autoreleasepoolto do so in an iterative logic helps reduce memory peak.

So *errorin -enumerateObjectsUsingBlock:the release out in advance, so then read *errorwhen there will crash.

Xcode 10 will give targeted static analysis warnings:

The correct way of writing should be like this:

- (void)findProblems:(NSArray *)arr error:(NSError *__autoreleasing*)error { 
	__block NSError *localError;
	[arr enumerateObjectsUsingBlock:^(id value, NSUInteger idx, BOOL *stop) {
		if ([value isEqualToString:@"problem"]) {
			localError = [NSError errorWithDomain:@"domain" code:1 userInfo:nil];
		} 
	}];
	if (error) {
		*error = localError;
	} 
}

Note: In fact, as early as last year WWDC17 Session 411 What's New in LLVM in Xcode 9 to be displayed on the introduction of a written __autoreleasingwarning.

Improved performance and visualization reports

The static analyzer in Xcode 10 can work in a more efficient manner, with an average 15% increase in the number of bugs found in the same analysis time.

Not only is the performance improved, Xcode 10 has also made improvements in the visualization of reports. There is an unnecessary and lengthy Error Path in the static analyzer report page of Xcode 9:

It has been optimized in Xcode 10:

personal opinion

Well~ For Xcode's static analysis, I personally think it is better than nothing. However, it is not recommended to do static analysis every time the project is built, which greatly increases the cost of building the project.

Personally, I suggest that you do static analysis before submitting the code to the small partners in the group after the self-test in the development process. This can avoid the emergence of some issues and find some hidden code hazards. Some problems can be exposed before submitting the code using a static analyzer, and there is no need to consume the valuable human resources of Code Review in the group.

You can also run a static analysis at regular intervals in the CI settings, generate reports and send them to small groups in the group, and assign the responsible person according to the problem to check whether it needs to be repaired (static analysis is not necessarily accurate under a more complex code structure ), so that regular maintenance can maintain the health of the project code from a certain perspective.

Increase safety

Stack Protector

Before introducing Stack Protector, Apple engineers carefully led the developers present to review the basic knowledge of Stack:

As shown in the figure above, it is actually a simple talk about the working method of Stack, such as the structure of the stack frame and the unwinding of the stack when the function is called. Each level of method call corresponds to a related activity record, which is also called an activity frame . The call stack of a function is composed of a frame structure, so it is also called a stack frame .

We can see that the stack frame contains the Return Address , which is the address to be returned after the execution of the current active record.

So what are the security issues? Apple engineers then introduced some privilege escalations achieved by modifying the Return Address of the stack frame by improper means. Well - that is a long history of buffer overflow attacks .

When using some unsafe functions in the C language (such as the above figure strcpy()), it may cause buffer overflow.

Note: The strcpy()function copies the source string to the specified buffer. But Ya didn't specify the specific number of characters to be copied! If the source string happens to come from user input, and its size is not specifically limited, it may cause a buffer overflow !

In response to buffer overflow attacks, LLVM introduced an additional area (the green area in the figure below) as a moat for the return address of the stack frame , called Stack Canary, which has been enabled by default:

Note: Canary is translated as "Canary". The name of Stack Canary comes from the fact that early coal miners would carry canaries when they went to the pit to detect whether the carbon monoxide in the pit reached a dangerous value, so as to determine whether they needed to escape.

According to our analysis of the principle of buffer overflow attacks above, you should easily find the defense principle of Stack Canary, that is, buffer overflow attacks are designed to use buffer overflow to tamper with the return address of the stack frame. After adding Stack Canary, you want to tampering Return Address stack Canary bound to pass, after the end of the current stack frame to be used to detect whether the implementation stack Canary Return Address changes from time back, if you call the abort()Force quit.

Well~ Does it look like a canary in a mine?

However, Stack Canary has some limitations:

The Canary area can be calculated and the value of the Canary area can be disguised during a buffer overflow attack, so that the Return Address is tampered with and the content of the Canary area remains unchanged, bypassing the detection.
Then a little rude words, by double strcpy()data from any memory protection override, by constructing an appropriate overflow string can be achieved GOT (Global Offset Table) modified ELF (Executable and Linking Format) mapping, as long as the modified GOT the _exit()entrance, even if the Canary detected tampering, before calling the function returns abort()quit or will go has been tampered with in _exit().

Stack Checking

Stack Protector is an existing feature of Xcode and turned on by default, and Stack Checking is a new feature introduced by Xcode 10, mainly for Stack Clash issues.

The Stack Clash problem arises from Stack and Heap. Stack grows from top to bottom , while Heap grows from bottom to top . The two expand towards each other and memory is limited.

Works Stack Checking is specified in the area Stack reasonable dividing line (red line on the chart), the internal function of the variable length of the buffer to do check the buffer size to be allocated, if the buffer beyond the dividing line is called abort()Force Quit .

Note: LLVM team in this WWDC18 join Stack Checking, because of the high probability middle of last year Qualys to a published report on Stack Clash of .

New instruction set extension

Emmmmm .... This section is for the iMac Pro and iPhone X to use set architecture instructions - (ISA Instruction set architecture) expand done. Frankly speaking, I am not very interested in this area, nor have I studied it in depth, so I won t be ugly...

summary

This article sorted out the content in WWDC18 Session 409 What's New in LLVM, and shared my personal humble opinion on these content. I hope it will be helpful to all the students who have not had time to watch WWDC18 Session 409 for various reasons.

The article is written carefully (it is my own original article, please indicate lision.me/for reprinting ). If errors are found, they will be updated in my personal blog first. If you have any questions, please contact me on my Weibo @Lision ~

I hope my article can bring you value~

WWDC18 What's New in LLVM

WWDC18 What's New in LLVM

Preface

index

ARC update

personal opinion

Xcode 10 new diagnostics

Swift and Objective-C interoperability

personal opinion

Use #pragma packpackaged Struct members

personal opinion

Clang static analysis

GCD performance anti-pattern

The auto-release variable exceeds the auto-release pool

Improved performance and visualization reports

personal opinion

Increase safety

Stack Protector

Stack Checking

New instruction set extension

summary

Use `#pragma pack`packaged Struct members